DIGITALLY AUGMENTING PHYSICAL SPACES FOR PERSONALISED LEARNING

Transcription

1 DIGITALLY AUGMENTING PHYSICAL SPACES FOR PERSONALISED LEARNING Abstract Julie Boston, Martin Masek, Mark Brogan, Chiou Peng Lam Edith Cowan University, Perth, Western Australia The ubiquity of mobile computing devices, such as phones and tablets has led to their increased use in education. The ability of these devices to augment physical spaces with additional content shows particular promise to enable the creation of guided and learner driven learning experiences. In this paper, we introduce Trailblazer, a software framework that allows non-programmers to create rich augmented reality experiences. The framework allows for activities that are composed tasks that include a mixture of information transmission and knowledge testing through a multi-modal experience. A case study is presented demonstrating the framework in the cultural heritage space with Year 5 integrated curriculum. To assist the learning process it can be helpful to journey out of the classroom and into certain places of interest. For example, visiting a heritage site to learn about the past or attending a significant event, such as eruption of a volcano. There are, however, barriers to such excursions. The place of interest could be impractical to get to, dangerous, or the significant event may have already passed. Further, if the site is visitable, expert guidance is needed in taking a group through the site. A promising solution to the barriers on physical excursions is to augment them with mobile-based augmented reality applications (MAR). In augmented reality (AR), a physical place is transformed by adding virtual content using the mobile device. This is typically implemented by using the camera on the mobile device to capture a view of the environment, presenting it on the screen with the additional virtual content superimposed on the actual scene. The content can include written text, images, video and recorded voiceover through to interactive 3D objects that appear to be a part of the actual scene. This has several applications. For one, an actual place of interest can be augmented with guides and information. This information can be directly matched to meet curriculum objectives so that the experience is both engaging and educationally effective. Further, events from the past and objects no longer present can be recreated through augmented reality. In addition, a space that is not connected to the place of interest, but is easily accessible (such as a school oval), can be transformed with augmented reality to represent the place of interest. Currently, there is no easy way to create a rich, engaging educational experience using AR. One current solution is to commission custom AR apps, typically a costly, time consuming process, resulting in an app where the content is not easy to change. Another solution is to use one of the existing consumer-accessible authoring tools, such as EveryTrail (2014) or Aurasma (2014). Such tools are currently limited to serving content rather than letting the user take an active part in the experience, or having the ability to scaffold learning tasks to build competence. In our work we address these limitations through an easy to use framework, Trailblazer, where rich experiences can be built from a set of generic task types that promote exploration and actively engage the participant. The rest of this paper is organised as follows. First, we provide a review of existing work in the augmented reality space, focusing specifically on learning. We then provide an overview of our solution, followed by a case study of applying our solution in the cultural heritage space. This is followed by a discussion of the feedback we have received so far and the conclusion. Page 1 of 9

2 Theoretical Framework and Literature review Innovation in learning with technology depends on individual access to innovative technologies and the pedagogy and infrastructure that support them. Contemporary theories of learning with information communications technology (ICT) emphasise socio-cultural and constructivist theories of learning (Webb & Cox, 2004). Pedagogical approaches to technology integration support studentcentred environments that are characterised by their focus on active participation, collaboration, and knowledge production rather than knowledge acquisition (Keengwe & Onchwari, 2011). One of the newest technologies currently being explored for its potential to offer interactive multimodal learning experiences is AR technology (Billinghurst & Dunser, 2012). Physical manipulation of digital content using AR technology is now possible thus opening up a myriad of opportunities for educators to assist learners in exploring abstract spatial and temporal concepts. How the coexistence of virtual objects and real environments might assist learning is the subject of a rich discourse. Arvanitis et al. (2007) argue that the coexistence of virtual objects and real environments allows learners to visualize complex spatial relationships and abstract concepts. According to Klopfer & Squire (2008), such coexistence enables the learner to experience phenomena that are not possible in the real world and to interact with two and three dimensional synthetic objects in a mixed reality environment. Both create a context for deep learning. Other researchers (Squire & Jan, 2007; Squire & Klopfer, 2007) argue that such environments enable the development of important practices and literacies that cannot be developed and enacted in other technology-enhanced learning environments. In a literature review of research in AR applications in education, Wu et al. (2013) state that such benefits make AR one of the key emerging technologies for education over the next five years. There is still much to learn regarding how AR can best be used for educational purposes (Folkestead & O Shae, 2011). Some recent projects suggest the possibility of AR complementing current learning models and tools. AntarticAR (Lee, Dunser Nassani & Billinghurst, 2013) investigated the application of AR to create a virtual tour of Antartica where significant portions of the real world (such as a school oval) were replaced with virtual content allowing the user to become immersed in Antarctica s extreme environment. The Handheld Augmented Reality Project (HARP) (O Shea et al. 2009), investigated the efficacy of AR curricula for engagement and understanding. This collaborative project developed two scenario-based AR curricula, targeted towards Massachusetts state academic standards for middle school math and languages. Prohibitive technological, management and cognitive overload issues were common across these projects; however, overall these projects demonstrated the potential usability of AR technology for engagement and understanding. Introducing Trailblazer Our solution in the AR space, Trailblazer, is targeted towards the goal of allowing non-programmers to build engaging augmented reality experiences. In line with this aim, we have opted to develop a graphical user interface (GUI) based system through which such experiences can be created and stored on a server. The content stored on the server is then experienced through a mobile app. The features of Trailblazer were determined using an iterative and incremental methodology using a participatory design process. In this approach, the system is built in increments of functionality with each undergoing several iterations. Feedback on iterations was sought from a wide range of stakeholders, as detailed in the next section on the case study. Due to the wide range of experience with mobile devices in the potential target audience, it became important to design a simple, intuitive interface. Some general principles used in the interface design of the app include: minimal options on each screen, large meaningful icons and Page 2 of 9

3 organization of the interface to support one hand interaction. In addition, as it is sometimes useful to situate AR activities outdoors, a colour scheme was chosen to maximise the contrast of user interface elements. The architecture of the Trailblazer system is shown in Figure 1. From the perspective of a learner (or anyone wishing to partake in the experience), they download a Trailblazer app onto their mobile device from an app store (such as itunes or Google Play). The app does not come with any AR content when initially downloaded, but provides a means of connecting to the Trailblazer server, which holds a set of experiences that the player can choose from to download and experience through the Trailblazer app. With this approach, the same app can be used for multiple learning experiences, rather than having to download a different app for each. This promotes a consistent experience in terms of the user interface and functionality for various types of learning experiences. Download app App Store Mobile Device Trailblazer Server Download content for app. Create and edit content. Web Browser Learner Author The Trailblazer framework from the perspective of both a learner and the author of the learning experience. From the perspective of an author, they build a particular experience on the Trailblazer server through a web browser interface. As we envisage Trailblazer to be used to create a wide variety of experiences, encompassing different content across different locations a flexible structure was needed in which generic experience elements could be placed. This structure, consisting of five levels, is shown in Figure 2 along with the specific experience built for the case study, described later in this paper and possible extensions. At the top level is the trail, which consists of one or more physical places of interest (POI). Each POI can have a number of quests, which are intended to be thematic groupings of activities based at that place of interest. An activity is made up of a number of tasks, each task being in a concrete location at the place of interest. These task locations are anchored either by GPS location or visual marker. Multiple sequential tasks at the same visual marker are supported, with the completion of one task causing the next task to appear. Page 3 of 9

4 Trail Swan River Heritage Quest 1..* Place of Interest 1..* Tranby House Explore Activity Task (a) 1..* 1..* Black Swan Arrival Task 1 Task 1 Task 2 Task 3 (b) Figure 2: The five levels of content in the Trailblazer framework (a), along with an example instance (b). Activities in a quest can be attempted in any order, and the author of the experience can create a combination of ordered and unordered tasks within an activity. Several task types have been created to allow the author to create the activity, summarized in Table 1. Table 1: Types of tasks present in the Trailblazer framework. Task Type Learner interaction involved Role Information Panel Presents a panel of text over a visual marker. Information relevant to the activity can be presented to the learner, such as clues to find the next task or Video 3D Model A video is played over a visual marker. The video pauses when the marker moves off-camera and resumes when the marker is visible again. A 3D geometric model is shown with position, orientation and scale to match the marker. The learner can manoeuvre around a virtual object is if it was in the physical environment. Key Provides an object that can be collected, anchored to a visual marker. The author can make other tasks unavailable (locked) until a set number of these objects have been collected. Multi-Choice Question The learner is provided with three possible answers to a question, one of which is correct. The learner selects an answer. Points are awarded for selecting the correct answer, with information about the POI. A more multi-modal experience than the text-based information panel. The video task type can be used to demonstrate concepts to the learner. The learner can make observations on objects that are not at the physical location. This can be used to engage learners with artifacts that may have once existed at the place of interest, or those too valuable to be on display to the public. Introduces an explore mechanic where the learner needs to investigate their surroundings in order to unlock a particular task. Provides a point for reflection, where the learner must make a decision. This decision can be informed by other content at the POI, or the learner can guess. This means the player need not Page 4 of 9

5 Written Answer Question points diminishing with the number of attempts before the correct answer is selected. Written text is provided (typically intended to be in the form of a question), with blank spaces that the learner has to fill in with a particular set of words. The author can elect for some of the letters in the answer to be revealed. be stuck on this task, however subsequent tasks should not assume knowledge of this question. A point where the learner can reflect, but harder than the multi-choice questions in that there is limited scope for guessing. The learner must enter the correct response in order to proceed with the activity. Tasks subsequent to this can assume the learner understands the question. In addition to the information provided directly with the task, each task can have documents attached to it that are added to the Items library on the Traiblazer app. These documents can be in the form of videos, documents (eg. pdf files), or images. Appropriate documents can be selected to help the user complete the activity, or to act as reference points for work done outside the app, such as reflective activities back at school for a school-excursion. Case study: Trailblazer for the cultural heritage space Working in partnership with the project sponsor, National Trust of Australia WA (NT), we have created a demonstration of the framework that delivers primary level integrated curriculum at Year 5 level, based around Western Australia s oldest surviving domestic building, Tranby House (TH). Curriculum integration refers to curricula that are aimed at making subject matter more relevant to students experiences with less focus on discipline boundaries (Gehrke, 1998). Though literature highlights barriers to integrated curriculum, such as teachers lack of knowledge outside their own discipline and difficulties in fitting established assessment systems (Lam et al., 2013), Trailblazer offers the architecture to embed knowledge from multiple disciplines and guide the students through the AR experience. Toward this end, we have developed two activities, Arrival and The Black Swan, that include content related to: environment, heritage, sustainability, history, literacy, and numeracy set in the context of the TH locality, aligning to meet the outcomes of the Australian History, Geography, Science and Mathematics Curriculum. Arrival activity The arrival activity consists of a series of non-linear puzzles associated with the historical artifacts located in the entrance foyer of TH. To encourage students to explore, they are first required to collect a number of AR keys hidden in the foyer, followed by alternating sequences of information panels and questions focused around artefacts with information about the journey the settlers of TH took to arrive in Australia and how TH was started. Challenges include having to calculate the month of arrival, examining the goods that the settlers bought with them against importance for survival and examining the size and shape of land allocated to the settlers. A screenshot from one of the tasks in the Arrival activity is shown in Figure 3 (a). Page 5 of 9

6 (a) (b) Black Swan activity This activity was situated outside TH, next to the banks of the adjacent river and positions students in the role of a biologist. A visual marker, given to students outside TH, triggers this AR experience which includes a 3D model of a black swan, screenshot shown in Figure 3 (b), accompanied by an audio narrative detailing the historical context and significance of the black swan as well as instructions for an inquiry task. Working in pairs the students must then orientate the tablet to gain different spatial perspectives of this species so that they may draw its key morphological features on the evidence worksheet. Ultimately this inquiry-based task will challenge students to collect evidence and form an hypothesis related to why so few of these birds exist now in this location. Here we are using AR to amplify the real world environment, as the likelihood of observing this particular indigenous species is very low due to destruction of its natural habitat and the level of watercraft activity on that particular stretch of the river. Results and Discussion Prior to testing with Year 5 students, formative feedback on the Trailblazer architecture and the Arrival activity was sought via a think tank of stakeholders, including representatives of the NT, teachers and curriculum experts and representatives of the indigenous community. The think tank was situated on site at TH, giving participants an opportunity to engage in an authentic user experience of the activities. A summary of this formative feedback, along with modifications made in response follows. Page 6 of 9

7 Our stakeholders perceived the navigation between the various visual markers used by the arrival activity was not intuitive and would potentially lead to cognitive overload. Consequently the instructional clues were re-written in a simpler and more targeted fashion. Navigational and user hints, including how to hold the tablet to trigger the AR experiences, were also added. Contrary to the evaluative feedback arising from the HARP project (O Shea et al. 2009), the non-linear nature of the arrival tasks seemed to invoke a sense of confusion. Suggested improvements included numbering the tasks or a screen tool added to indicate how much of the quest had been completed. On the visual presentation of tasks, participants indicated perceptual difficulties as the size of some markers (such as info panels) was scaled with distance away from the visual marker. This necessitated modifications to the presentation of makers, increasing their space to fill the screen and increasing the text font size. A further improvement that was made to the activities was to simplify the readability of the textual information to align more closely with the typical reading age of a Year 5 student. Further feedback specific to this POI was to feature more historical events or objects no longer present as part of the experience. A more multimodal experience, featuring video and audio was also suggested. The Black Swan activity was created in response to these suggestions. Value was also seen in students being able to capture video and photographic evidence of the completed quests, which then could be used back at school. This feature was seen to be innovative in terms of enabling further opportunities to conduct inquiry based activities with evidence collected on site. Various solutions exist to enable this, such as the use of cloud-based storage that the students could access from anywhere. This will be explored in future work. After the improvements in response to think tank feedback were made, the next iteration of Trailblazer and the activities were tested with the target audience, 32 Year 5 students from an Independent metropolitan primary school. The students were partnered up to complete both the arrivals and black swan activities. Almost without exception these ten-year-old students intuitively were able to use the swipe features of the tablet and without instruction understood that pressing the home button would return them to the beginning of the quest. High engagement in this activity was observed; however, unexpectedly issues arose with some pairs of students experiencing ownership issues over the single tablet device, which resulted in the NT tour guides and researchers having to strongly reinforce taking turns. The confined nature of the entrance foyer where the arrival activity was located resulted in unanticipated management issues requiring intervention by the NT tour guides and classroom teacher. Almost immediately, the researchers noticed a competitive element arising between pairs of students, with many students seen rushing to locate all the AR keys and solve the challenges. Once the students had been situated outside of TH next to the banks of the river gasps of delight were heard as the students triggered the appearance of the 3-D model of a black swan. Activating the AR black swan automatically triggered an audio introduction and set of instructions for this task. As these tablets were not purposefully designed for outdoor use, listening to this information was problematic due to the ambient background noise and the excited student chatter. The researchers intervened after noticing this difficulty and offered a verbal explanation for this task instead. Despite being instructed to remain seated during this drawing activity the students stood up in order to orient the tablet to achieve a complete view of this bird. Spatially this was a challenging task requiring the students to simultaneously view the black swan and attempt to draw its features. Without explicitly being instructed, the majority of students collaborated with their partner to successfully draw the black swan s key morphological features on the evidence sheet provided. A number of students were observed to be so deeply engaged with viewing the black swan that they became disoriented and were asked to sit down for fear of tripping or bumping into one another. The issue of glare on the screen was resolved in this particular play testing trial by situating the students in a shaded location, thereby maximizing screen contrast and clarity in this outdoor based activity. Page 7 of 9

8 Conclusion and Future Work Designing curricula specific learning activities that leverage off the affordances of AR is a continuing topic of research. In this paper, we have described the Trailblazer framework which we have designed to allow non-technical experts to craft such learning activities in order to accelerate research in the field and also to minimise barriers to practical implementation. The framework includes a web-based editor where the authors of experiences construct sequences of tasks, having the ability to interweave text-based information with 3D virtual models, videos as well as questions to encourage the learner to reflect and engage with the material. To demonstrate the framework we have created two demonstration activities and received promising feedback from the Year 5 audience. In future work we are aiming to explore a wider set of activities and also to enhance the framework with a wider array of task types. Acknowledgements This work was supported by an Edith Cowan University Industry Collaboration Grant with The National Trust of Australia (WA). References Arvanitis, T. N., Petrou, A., Knight, J. F., Savas, S., Sotiriou, S., Gargalakos, M., et al. (2007). Human factors and qualitative pedagogical evaluation of a mobile augmented reality system for science education used by learners with physical disabilities. Personal and Ubiquitous Computing, 13(3), Aurasma (2014), Retrieved 22 April 2014 from: Billinghurst, M., & Duenser, A. (2012). Augmented reality in the classroom. Computer, 45(7), doi: /mc EveryTrail (2014), Retrieved 22 April 2014 from: Folkestad, J., & O'Shea, P. (2011). An analysis of engagement in a combination indoor/outdoor augmented reality educational game. Journal on School Educational Technology, 7(1). Gehrke, N. J. (1998). A look at curriculum integration from the bridge. Curriculum Journal, 9(2), Keengwe, J., & Onchwari, G. (2011). Fostering meaningful student learning through constructivist pedagogy and technology integration. International Journal of Information and Communication Technology Education, 7(4), doi: /jicte Klopfer, E., & Squire, K. (2008). Environmental detectives: the development of an augmented reality platform for environmental simulations. Educational Technology Research and Development, 56(2), Lam, C. C., Alviar-Martin, T., Adler, S. A., & Sim, J. B. Y. (2013). Curriculum integration in Singapore: Teachers' perspectives and practice. Teaching and Teacher Education, 31(0), doi: Lee, G. A., Dunser, A., Nassani, A., & Billinghurst, M. (2013). AntarcticAR: An outdoor AR Page 8 of 9

9 experience of a virtual tour to antarctica doi: /ismar-amh Martin, S., Diaz, G., Sancristobal, E., Gil, R., Castro, M., & Peire, J. (2011). New technology trends in education: seven years of forecasts and convergence. Computers & Education, 57(3), O Shea, P., Mitchell, R., Johnston, C., & Dede, C. (2011). Lessons learned about designing augmented realities. Int l J. Gaming and Computer-Mediated Simulations, (1), Squire, K., & Jan, M. (2007). Mad city mystery: developing scientific argumentation skills with a place-based augmented reality game on handheld computers. Journal of Science Education and Technology, 16(1), Squire, K., & Klopfer, E. (2007). Augmented reality simulations on handheld computers. Journal of the Learning Sciences, 16(3), Webb, M., & Cox, M. (2004). A review of pedagogy related to information and communications technology. Technology, Pedagogy and Education, 13(3), doi: / Wu, H., Lee, S. W., Chang, H., & Liang, J. (2013). Current status, opportunities and challenges of augmented reality in education. Computers & Education, 62, doi: /j.compedu Page 9 of 9