A PORTABLE AUGMENTED REALITY LAB

Transcription

1 176 1st International Malaysian Educational Technology Convention A PORTABLE AUGMENTED REALITY LAB D.R.Awang Rambli, 1 S.Sulaiman. 2 M.Y.Nayan 1 Dept of Computer and Information Sciences Universiti Teknologi PETRONAS, Tronoh, Perak, Malaysia {roharam, suziah}@petronas.com.my 2 Dept of Electrical and Electronics Engineering Universiti Teknologi PETRONAS, Tronoh, Perak, Malaysia yunusna@petronas.com.my, ABSTRACT The importance of lab-based courses and its significant role in the science education is irrefutable. It has been universally accepted that it could enhance the understanding of science concept through hands-on experience. Several researchers have also asserted its strong impact on the students learning outcome. However, the economic constraints of creating and maintaining traditional science laboratories in schools have resulted in only a few labs allocated per school. Thus, large students number per school and large class sizes may result in students not fully benefiting from the effectiveness of laboratory education. The recent years have seen the transformation of laboratory education through the introduction of simulated lab concept using information technology as an alternative to the traditional lab. This paper provides a conceptual framework of a simulated but portable augmented reality lab which could be used in teaching of science in classrooms. Augmented reality is a promising technology which allows seamless user interaction between the real and virtual objects. The aim of the project is to enhance the teaching and learning of science by complementing the existing traditional lab with the use of a simulated augmented reality lab. The system architecture and the technical aspects of the proposed portable augmented reality will be described. Implementation issues and benefits of the proposed system will also be highlighted. INTRODUCTION Hands-on laboratory experience is an important element in the science education. Noresessian (1991) claims that hands-on experience is the heart of science learning, whilst Clough (2002) argues that it make science learning come alive. Laboratory sessions allow students to see, feel and hear of what have been described in textbook or lectures (Bhargava et al, 2004). They provide students the opportunities to investigate, test hypothesis, and perform data analysis in order to find correlation between data and variables. However, due to economic constraint of creating and maintaining traditional science laboratories in schools only few labs are allocated per school. In Malaysia, this is inadequate considering the large students numbers per school and large class sizes. Thus the full benefit of laboratory education may not be fully realized. Advances in technology together with affordable cost had enabled the use of innovative teaching and learning tools for education. Virtual Reality (VR) a technology which was once expensive and limited it used as simulators for training in areas such as military, aerospace and etc can now be found used in education. Recently, Augmented Reality (AR) which is a variation of VR had been used in education and had demonstrated high potential to enhance students learning experience. The recent years have witnessed the changes in the science laboratory education scenes through the implementation of simulated or virtual laboratories (e.g Dede 1996; Byrne 1996; Roussou et al 1997; Zagoranski & Divjak 2003; Subramaniam & Marsic 2001). Such implementations include the use of virtual reality and multimedia concepts. Latest development involves augmented reality mainly because of its capability in supporting user interactions between the real and virtual objects at the same time. This characteristic enhances users performances as the interaction becomes more realistic and intuitive (Chen 2006).

2 177 1st International Malaysian Educational Technology Convention In this paper, a conceptual framework of a simulated but portable augmented reality lab is presented. The intention is to propose an alternative to a traditional lab for teaching and learning science. The system architecture and the technical aspects of the proposed portable lab are described. A discussion on the implementation issues and benefits of the proposed system are provided. BACKGROUND Augmented Reality (AR) is a medium which overlays virtual objects on top of a person s local real world environment (Chen 2006). It is a new technology that generates three-dimensional (3-D) virtual objects, and provides an interactive interface with which people can work and interact simultaneously both in the real world and 3-D virtual objects. Despite AR being a new technology, it has been applied to many areas such as medical, military services, architecture and entertainment. For example, in the medical field, AR visualization has been suggested and investigated for ultrasound imaging (Bajura et al 1992; State, Chen et al. 1994) and image guided surgery (Lorensen, Cline et al. 1993; With respect to entertainment, AR technology has been used to create special effect for creating illusion (Pyros and Goren 1995) and to enhance gaming experience through the development AR games (Liarokapis 2006). Even though AR could be applied to various domains of applications, the technology is projected to have a more significant role in teaching and learning since it is capable in immersing individuals in experiential learning environments. Among the main features identified in AR that assist in the conceptual learning include: ability to draw people s attention due to the technology being newly introduced; tendency to create a constructivist environment; supporting both the creation of visual images and conveying the spatial cues directly to users; and allowing users to interact with the system by using their body, especially the hands (Chen 2006). These features provide a justification to employ AR as a tool for teaching and learning abstract concepts in science education. RELATED WORK Although the concept of AR has been implemented in many areas of application including education in the other parts of world, the idea is relatively new especially in the field of education. Two closely related concepts; multimedia and virtual realities have been explored and implemented, especially the former, in numerous educational research projects. A significant number of VR applications have been developed for the teaching of science subject, ranging from primary to tertiary education. Some of the earlier VR project for science education includes the NICE project (Roussou et al 1997), Water on Tap (Byrne 1996), and Science Space (Dede 1996). The NICE project was designed for children aged 6 to 10. It allows the children to grow plants by manipulating variables such as water and light in the virtual world. This project explored the potential of VR as a learning medium. Water on Tap is a chemistry world where students learn about the concepts of molecules, electron and atoms. On the other hand, ScienceSpace project allows students to investigate concept of Physics. It is a collection of several virtual worlds: Newtonworld, MaxwellWorld and Pauling World. Rather than focusing on VR, some researchers have investigated AR as a learning medium. Zagoranski & Divjak (2003) explore and implement AR in creating experiments based on abstract models. They argued its usefulness in terms of personalized learning without real experiment equipment. However, students complained of not being able to experience the real environment. Liarokapis et al (2002) present an innovative use of AR interface for engineering education to support collaborative leaning where students developed teams skills. The aim of the project was to enhance students learning and understanding of digital design using commercial design flows. In a more recent research study, Chen (2006) investigated AR utility for concept learning in chemistry education by comparing students perception of AR and physical models of amino acids. Study s results show that students like manipulating the AR model but still prefer the

3 178 1st International Malaysian Educational Technology Convention physical model to get the feel of it. Surprisingly, students tend to treat the AR model as real objects. MOTIVATION FOR THE RESEARCH Some researchers suggest that AR could enrich educational experience particularly the young ones (Billinghurst 2002; Camarata et al. 2002; Schnadelbach et al. 2002). Others have claimed that AR enhance user experience (Underkoffler & Ishii 1998; Chen 2006) and improve collaborative works (Kiyokawa 2002; Fjeld et al., 2002). AR application interface is different from virtual reality or multimedia in that it enhances the real experience by allowing users to concurrently view the real world with the virtual image. The virtual image could be in the in the form of text image, static model or animal model, could be additional information provided to the user to enhance his knowledge. By capitalizing on our familiarity with the everyday physical world, AR applications allows user to interact with computer in a similar way as we do in the real world (Yvonne et al 2002). Unlike the traditional method of interaction using mouse and keyboard, the AR interface metaphor is based on user using the physical objects such as cards to interact with virtual objects in natural way (Bilinguirst 2002). Being different from the traditional interface method, this will naturally grab the student attention (Chen 2006). Additionally, the use of AR not only led to high motivation in student but its intuitive and user-friendly interaction method could led to a better understanding of the concept taught. To educate students is a challenging endeavor; getting their attention, engagement and motivation in learning process are some of them. Thus, incorporating this interesting AR technology as part of the educational delivery method would be a great alternative for students. Previous researchers have shown the potential of AR in the learning of science (Chen 2002; Shelton & Hedley 2002; Kerawalla et al 2006). Very little is known on how AR can be incorporated in the classroom environment (Kerawalla et al 2006). As such, our research project would like to explore its potentials in the teaching of science in classrooms, particularly to supplement the existing traditional lab through the use of our concept of simulated portable augmented reality lab. Moreover, the current educational scenarios in Malaysia which emphasize the use of technology particular computers provide the necessary infrastructure for implementing this project idea locally. Some schools are already equipped with computer labs, laptops for teachers, and LCD projectors. However, if one is not available, it is necessary to acquire web cameras, one of the necessary components of our proposed system. Informal Study on Users Perception and Acceptance towards AR Since AR concept is relatively new, especially in Malaysia, an informal study was conducted to examine users perception and acceptance towards this technology. A simple prototype AR game application was developed for this purpose (Figure 1). Ten students, who have no prior exposure to AR game experience, were first asked to play the game and later were asked to answer a short questionnaire based on their experience. Results show that a majority of the students reported they were greatly impressed by this new concept of playing with the computer. Some indicated that the concept of overlaying virtual objects over real objects was very interesting. Most agreed that they like the different way of interacting with the computer. These preliminary results indicate the potential of an AR application among students; thus provide further motivation towards the development of AR lab project to examine the application of AR concept in science education.

4 179 1st International Malaysian Educational Technology Convention Figure 1: Prototype AR game application CONCEPTUAL FRAMEWORK Reviews from the earlier sections suggest the potential of AR in science education. Realizing these, the following framework (see Figure 2) is suggested for integrating and implementing AR lab into the teaching and learning of science. Science concept /experiment Internet/home/ computer lab {virtual experiment for 1. practice/revision 2. Homework/assignment AR lab {computer notebook, LCD projector, pattern markers, Projector screen} Classroom {virtual experiment for 1. Practice -Prior to actual 2. Experiment not feasible due to - Costly - Safety reason - take more than lab hours - Limited resources, etc} Traditional lab {uses prior knowledge from virtual experiment to conduct actual experiment} Figure 2: Conceptual Framework for integrating and implementing the portable AR lab into the teaching and learning of science Unlike the traditional science laboratory, the portable AR lab does not require a physical space or special room. The main idea behind portable AR lab system is teachers can bring the AR application in their laptop or notebook to the classroom along with an LCD projector and the required pattern markers. If no projector screen available, one of the classroom walls could be used as a substitute. In the classroom, the teacher can demonstrate virtual experiment to the whole class. The class session does not have to be limited to one way communication only. Depending on teaching creativeness, this can be interactive class sessions where students could be asked to participate either verbally or can be asked to try out the experiment by manipulating the pattern markers. The AR lab could be used to prepare students prior to conducting the actual experiment. Students can practice and understand the concept first. This can save a lot of time later in the lab and avoid students making unnecessary errors especially when resources are limited. Some experiments which are costly, cannot be conducted within the normal given lab time or due to safety reasons can still be carried out solely and virtually through the AR lab systems to

5 180 1st International Malaysian Educational Technology Convention illustrate concepts, test processes and ideas. Thus, students still can benefits the hands-on experience virtually. If deployed over the internet, the AR lab could be accessible from their home or the school s computer labs. The student needs to download and print the pattern markers. With the use of a web cam to capture pattern marker and a computer monitor as a display, student can do the virtual experiment using the AR lab system own their own or under the guidance of their parent. They can practice as often as needed. Teachers could also give some experiments as homework or assignments utilizing the AR lab. FOCUS OF THE STUDY One of the most important features of this AR lab system compared to VR or multimedia applications is in the manner user interact with the computer. To illustrate this, experiment on factors affecting plant growth rate is used. Several virtual models of plant at different stages of growth and a set of patterns marker to represent factors water, light, food need to be created. A teacher or a student selects a factor by choosing the corresponding pattern. Patterns for water could be labeled as water given, no water given, less water and more water. When detected by the camera, the corresponding virtual plant is displayed on the video image of real object. Potential topics that could be explored using AR concept include food chain, electricity and solar system. It is important to note that the focus of this project is on the system design. The instructional design (ID) part of the application which requires the respective expert to provide input on this important aspect of education delivery is not included in the project investigations for the current project. Target group of users for the portable AR lab is primary school students, ages 9 to 12 years old. THE PROPOSED PORTABLE AUGMENTED REALITY LAB System Components The proposed portable AR lab system comprises of a database of virtual models, pattern marker cards with corresponding database of patters, web camera, laptop or notebook, LCD projector and projector screen (Figure 3). Database of virtual models Virtual models are superimposed on video images of real object Markers Webcam Database of LCD projector patterns Notebook Projector Screen/ Computer Monitor display Figure 3 Essential components of portable AR lab system System Architecture The proposed portable AR system is based on the working principle of the ARToolKit algorithm ( As illustrated in Figure 4, a database of virtual models is required to be developed. These models can be static or animated and each model can represent an object, picture, animation or text images. Pattern markers on cards are used by the user to interact with the computer. Each pattern design is unique and is associated to a virtual model. A database of these patterns needs to be stored in memory. The web camera is used to capture the images of the pattern marker cards. The captured image will be used by the

6 181 1st International Malaysian Educational Technology Convention ARToolKit search algorithm to search the database of patterns. When a match is found, the corresponding virtual object is displayed on the marker and viewed by the user on the projector screen or monitor display. Another type of display for AR applications is the see-through headmounted display (HMD). Although, this display device will result in better viewing of the displayed images, due to cost factor this type is not recommended. Moreover it is meant for a single user; having at least thirty HMDs, one for each student in the class is not practical. Thus, a computer monitor or projector screen is a better alternative. Laptop/Notebook Database of patterns Pattern marker cards WEB CAM Camera tracks patterns on card ARToolKit AR software Database of virtual models LCD projector Projector screen/ Computer monitor Augmented View Virtual image over Real image Figure 4: Augmented Reality system Software for System Development The software used for the development of portable AR lab is ARToolKit. ARToolKit is a software library that allows rapid development of Augmented Reality applications. This software was chosen particularly due its low learning curve for new developers. The virtual objects are modeled using VRML and 3D Studio Max software. A prototype AR game was developed to survey user perception towards this technology. The result of the survey is described in the following section. For the research project, prototype AR experiments are currently being developed. DESIGN AND IMPLEMENTATION ISSUES Prior to the use of the AR system, teachers and students need to know how to setup the system. For internet users, they need to download and print marker patterns (for first time use only). Viewing the virtual the objects are affected by several factors. The lighting conditions may affect image quality which may in turn influence the search algorithm to get the respective virtual image to be presented on the display screen. Positioning the pattern markers and the web cam is important to get the video capture of the patterns to send to the computer. Pattern size and pattern complexity need to be considered when designing the pattern. Larger pattern size allows larger distance for the pattern to be detected. Simple patterns can be detected more easily compared to complex ones. Not all topics are suitable to be implemented using AR. Thus, identifying appropriate topics and experiments to incorporate in the AR lab system is needed. AR is a visualization tool which augments static or animated virtual objects on the real objects. Careful discernment and creative use of these features is necessary to determine the right topics which could utilize AR technologies to enhance the learning process. Like any educational and training software, adding

7 182 1st International Malaysian Educational Technology Convention intelligence to the application to present various kinds of experiments scenarios and results would be an advantage. The system could be designed to response accordingly to students level of knowledge and experiments scenarios. Finally, the incorporation of ID knowledge to present the learning material is necessary to ensure the right teaching techniques and strategies are employed. CONCLUSION: POTENTIAL BENEFITS OF AR LAB TO THE TEACHING AND LEARNING OF SCIENCE Augmented reality, which overlay virtual image over real object, presents a new method of interacting with computer for the user. The virtual experiments conducted through the AR lab can be used as a means to test ideas not testable within the time, cost and safety constraints of the usual lab. Due to limited or costly resources, they can also be used to prepare a student before doing the actual experiment. Other advantage is it allows students to practice doing virtual experiment as often as they need without being present at the physical lab at their convenience. It is noted that virtual experiments is not meant to replace the real hands on experience in a real physical laboratory. Real hands-on experience would still be a better option. The portable AR lab is meant to complement and supplement the teaching of science in the classroom. Our conceptual framework extends its utility beyond the classroom to internet or online deployment. The reviewed literature and the encouraging responses as reflected in our initial study provide motivation for further research. Future works will focus on identifying appropriate topics for AR lab applications, implementing and evaluating the system. Several possible research questions to be explored include: Can AR lab be used to support science education? Particularly, can an AR lab be used to enhance the learning of science concepts by complementing the existing traditional lab? Can AR lab be used to motivate learner to learn science? Can AR lab be used to sustain learner s attention and engagement? It is expected this new interface technique to provide motivation and better understanding of science concept through visualization for students. We expect the unique way of interaction could make the teaching of science more interesting and engaging especially among young children. REFERENCES: Alavi, M., Wheeler, B. C.& Valacich, J. S. (1995). Using IT to reengineer business education: An exploratory investigation of collaborative telelearning. MIS Quarterly, : p Bajura, M., Henry F, & Ryutarou Ohbuchi. (1992). Merging Virtual Reality with the Real World: Seeing Ultrasound Imagery Within the Patient. Computer Graphics 26, 2 (July 1992), Bhargava, P, Antoakakis J, Cunningham C & Zehnder A. (2006). Web-Based Virtual Torsion Laboratory. Computer Applications in Engineering Education. 4(1), 1-8. Billinghurst, M. (2002). Augmented Reality in Education. December 2002 New Horizons for Learning. Available at Byrne, C. (1996). Water on Tap: The Use of Virtual Reality as an Educational Tool. Doctoral Thesis, University of Washington, College of Engineering, Washington. Dede, C., Salzman, M. C., & Loftin, R. B.. (1996).ScienceSpace: Virtual Realitie for Learning Complex and Abstract Scientific Concepts," Paper presented at IEEE VRAIS '96, pp Camarata, K., Yi-Luen Do, E., Gross, M.D., Johnson, B.R. (2002). Navigational blocks: tangible navigation of digital information. Paper presented at CHI 2002 Proc., Chen, Y. (2006). A study of comparing the use of augmented reality and physical models in chemistry education. Paper presented at VRCIA Hong Kong June 2006, Clough, M.P. (2002). Using the laboratory to enhance student learning. Learning Science and the

8 183 1st International Malaysian Educational Technology Convention Science of Learning, R. W. Bybee, Ed. National Science Teachers Association, Washington D.C., Fjeld, M., Lauche, K., Bichsel, M., Voorhorst, F., Krueger, H. and Rauterberg, M.(2002). Physical and virtual tools: Activity theory applied to the design of groupware. In B.A. Nardi & D.F. Redmiles (Eds.), A Special Issue of CSCW: Activity Theory and the Practice of Design, Vol. 11(1-2), Kiyokawa, K. Billinghurst, M. Hayes, S. Gupta, A. Sannohe, Y. Kato, H. (2002) Communication Behaviors of Co-Located Users in Collaborative AR Interfaces." Paper presented at IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR 2002), 30 Sept. - 1 Oct., 2002, Darmstadt, Germany, IEEE Press, Los Alamitos, CA, pp Kerawalla L, Luckin R, Seljeflot S, Woolard A. (2006). Making it real ; exploring the potential of augmented reality for teaching primary school science. Virtual Reality, 10, Liarokapis F. (2006). An exploration from virtual to augmented reality gaming. Simulation & Gaming, Sage Publicatioin, December, 37(4): Liarokapis F., Mourkoussis N., Petridis P., Rumsey S., Lister P.F., White M. (2002). An Interactive Augmented Reality System for Engineering Education. Paper presented at 3 rd Global Congress on Engineering Education, Glassgow, Scotland, UK Lorensen, W., H. Cline, C. Nafis, R. Kikinis, D. Altobelli & L. Gleason,(1993). Enhancing Reality in the Operating Room. Paper presented at the 1993 IEEE Visualization Conference, pages , Roussos, M., Johnson, A., Leigh, J. Vasilakis, C. Barnes, C. & Moher, T. (1997). "NICE: Combining Constructionism, Narrative, and Collaboration in a Virtual Learning Environment," Computer Graphics, vol. 31, pp Mantovani F. (2001).VR Learning: Potential and Challenges for the Use of 3D Environments in Education and Training. In Towards CyberPsychology: Mind, Cognitions and Society in the Internet Age, Giuseppe Riva & Carlo Galimberti (Eds.), Amsterdam, IOS Press, Nersessian, N.J., (1991). Conceptual change in science and in science education. History, Philopsophy, and Science Teaching, M.R. Mathews, Ed. OISE Press, Toronto, Canada, Piaget, J. (1967).Bilogie et Connaissance [Biology and Knowledge]., Paris, France: Gallimard. Pyros, G. G. & B. N. Goren. (1995). Desktop Motion Image Processing for Episodic Television. Advanced Imaging, 10(7): Rogers Y,, Scaife M, Gabrielli S., Smith H., Harris E (200). A Conceptual Framework for Mixed Reality Environments: Designing Novel Learning Activities for Young Children. Presence 11(6): (2002) Rose H., Billinghurst M. & Zengo Sayu. (1995) An Immersive Educational Environment for Learning Japanese.HITL Technical Report No. TR Schnadelbach, H., Koleva, B., Flintham, M., Fraser, M., Izadi, S., Chandler, P., Foster, M., Benford, S., Greenhalgh, C., Rodden, T. (2002). The Augerscope: A mixed reality interface for outdoors. Paper presented at CHI 2002, Minneapolis, Minnesota USA, Shelton B, Hedley N. (2002). Using Augmented Reality for teaching earth-sun relationships to undergraduate geography students. Paper presented at 1 st IEEE International augmented reality toolkit workshop. Darmstadt, Germany. State, A., D. T. Chen, C. Tector, A. Brandt, H. Chen, R. Ohbuchi, M. Bajura & H. Fuchs (1994). Case Study: Observing a Volume Rendered Fetus within a Pregnant Patient. Paper presented at the 1994 IEEE Visualization Conference, pages , Subraniam R.& Marsic, I. (2001). ViBE: Virtual Biology Experiments. Paper presented at WWW10, May 1-5, Hong Kong, Underkoffler, J & Ishii, H. (1998). Illuminating light: An optical design tool with a luminous-tangible interface. In CHI 98 Proc., Los Angeles, CA, USA, Vygotsky, L.S., (1978). Mind in Society: The development of higher psychological processes. Cambridge, MA: Harvard University Press. Zagoranski S, Divjak S Use of Augmented Reality in Education. Paper presented at EURON 2003, Ljubljana, Slovenia,