Car Advertisement for Android Application in Augmented Reality

Car Advertisement for Android Application in Augmented Reality Tan Siok Yee The National University of Malaysia Bangi, Selangor, Malaysia esthertan90@hotmail.com Haslina Arshad The National University of Malaysia Bangi, Selangor, Malaysia Waqas Khalid The National University of Malaysia Bangi, Selangor, Malaysia Abstract Many people find it difficult to imagine the real look and design of a car simply by looking at the pictures or posters advertised in the magazines and newspapers. The consumers do not have the opportunity to see different views of the car such as the rear, side and front view of the car due to their printed 2D nature. Therefore, this project aims to develop an application that can show a more comprehensive condition of the car to the consumers. The use of smart phones with android operating system as a delivery medium allows users to access this type of application anytime, anywhere. The 3D Perodua Myvi model is design using Blender software and export to.obj format. AndAR library is use to develop this application. By using this application, users can view different colors available in the market of Perdua Myvi car through their android smart phone. The users can also view the Perodua Myvi in a range of views including top, front, left, right, rear view and so on. The car can be rotated at 360 degrees due to the rotating abilities of this application. The users can use this application by scanning the marker provided in the advertising medium through the camera of the smart phone. Keywords: AndARToolkit, Android, AR Marketing, Mobile Augmented Reality I. Introduction The ultimate goal of Augmented Reality (AR) is to provide better management and ubiquitous access to information by using seamless techniques in which the interactive real world is combined with an interactive computer-generated world in one coherent environment. AR combines real world and digital data (Siltanen, 2012). It uses live video images, which the system processes digitally to add computer-generated graphics. In other words, the system augments the image with digital data. In the era of globalization, technology is rapidly growing from time to time. These emerging technological tools such as smart phones, tablets are now built in with cameras. This has attracted researchers to explore and develop applications based on AR (Juin, 2012). This paper is related to car adverting based on AR. This AR based car advertising in android environment can facilitate users to view 3D car model in different perspectives and views on top of the smart phone screen. AR based car advertising is an idea to attract the attention of consumers, especially for potential Proton Myvi buyers. AR based car advertising can convey information about the car to the consumer in an effective way. People cannot imagine a real car design when looking at advertising in newspapers or magazines. Users cannot see the situation of a car from a different view like rear view, side view or front view from the car advertising in the print media. Most of the people can only imagine the car through the photos posted in the print media. ISSN: 2289-2265 Page 80

Therefore, the objective of this paper is to create a Perodua Myvi design in threedimensional form, develop an AR application based car advertisers and create a marker that can generate 3D models of cars and displayed visually in reality environments. Perodua Myvi is the top selling car in Malaysia since year 2006 (Ramlee, 2011). By using this application, users can see the real situation of Perodua Myvi in different color available in the market. Users can move, change the scale and rotate the Perodua Myvi in 360 degrees. Users can also make screenshots of the Perodua Myvi using this application. This application named it as MyviAR Application. This application is developed using android operating system because it is an open source (Bagal and Kale). Android platform used in MyviAR Application is Android 2.3.3 (Gingerbread). MyviAR Application involves the design of 3D models Perodua Myvi using Blender software. This application is developed in Eclipse IDE using the AndAR library. The AR marker is created using online software called AR Marker Generator. 2. Literature Review 2.1 Augmented Reality (AR) In a nutshell, AR means combining virtual objects into the real world. However, AR still does not have a formal definition. Azuma et al. define AR as systems with three characteristics (Azuma et al., 2001): i. Combines real and virtual objects in a real environment; ii. Runs interactively, and in real time; and iii. Register (aligns) real and virtual object with each other. First feature of AR is to combine both the real world and the virtual world. Films such as Avatar which combined virtual objects into the real world cannot be considered as an AR because the film does not have interactive features. Fig. 1 shows the virtual reality continuum provided by Milgram et al. (Milgram et al., 1995). 2.2 Mobile AR Figure 1: Virtual Reality Continuum A mobile see-through display must be tracked with 6 degrees of freedom (6DOF) in order to display the correct image overlay. Tracking is a general problem in many applications, not only in AR but also in VR. Tracking is significantly more difficult in AR than VR due to the registration requirements. Registration is directly dependent on the tracking accuracy, making AR tracking become a very demanding task. Ideally, the resolution of both the tracking sub- ISSN: 2289-2265 Page 81

system and the display should be that of the fovea of the human eye (Wagner and Schmalstieg, 2009). In broad terms, tracking technology can be divided into mechanical, source-based, source-less and optical as shown in Table 1. Mechanical tracking systems calculate the final orientation and position by traversing armature limbs, accumulating the relative position of each link. They are accurate but cumbersome and with limited motion range. Type Technology Example Machanical Amature SensAble Phantom Sourcebased Magnetic, Ultrasonic Polhemus FASTRAK Optical Fiducial markers, natural feature tracking A.R.T.AR Track, ARToolKit Hybrid Optical-Inertial InterSense IS-I-200 Vis Tracker Table 1: Tracking technology examples The principle behind source-based tracking is to measure the distance between sources and receivers. To achieve 6DOF tracking, three sources and three receivers must be used. Electromagnetic trackers emit a magnetic field while acoustic trackers transmit ultrasonic signals picked up by microphones. Both these trackers are unobtrusive and industry-proven but can only be used in controlled environments. GPS and WLAN sources can be used for positioning only. Inertial trackers are source-less devices that measure change in inertia. Accelerometers measure acceleration and yield position data when their output is integrated twice. Gyroscopes measure angular motion and work by sensing the change in direction of an angular momentum. Gyroscopes require calibration while accelerometers use dead reckoning and are hence prone to drift over time. The big advantage is that both technologies can be miniaturized and also deployed in unprepared environments. Compasses give absolute heading relative to the Earth s magnetic field. However, they are vulnerable to distortion. Optical tracking is based on analysis of video input and calculates either absolute camera pose relative to known geometries or camera motion relative to the previous frame from extracted features. These geometries can either be 3D objects or 2D markers called fiducials. Alternatively, one can speak of marker-less and marker-based tracking. Optical tracking requires a clear line of sight and computer vision algorithms are computationally heavy (Abawi et al., 2004). There are however several strengths: it is cheap, accurate and flexible and one single sensor provides 6DOF tracking. It is very well suited for video see-through displays presented in the next section. Cameras are readily available in mobile phones where coupled with a CPU for possible image analysis. It should be noted that optical tracking extends to nonvisible wavelengths e.g. infrared. Very accurate range measurements can also be obtained by another form of optical tracking: laser beams (Henrysson, 2007). This work is focused on the optical tracking using fiducial marker for cheap, accurate and faster tracking. 2.3 Fiducial Marker Fiducial markers are commonly used for providing the tracking required by the AR applications. ARToolKit is example for the fiducial tracking libraries (Schall et al., 2005). ARToolKit works by thresholding the video frame to obtain a binary image in which connected regions are detected and checked for square shape. The interior of a candidate square is then matched against templates to identify it and to obtain its principal rotation. From the principal rotation and the corners and edges, the camera pose relative to the marker is calculated. Fig. 2 illustrates the pipeline. In a single marker setup, the world coordinate system has its origin in the middle of the marker. Additional markers can be defined relative to the origin marker to extend tracking range or they can represent local coordinates of a scene subset or interaction ISSN: 2289-2265 Page 82

prop. To be tracked, a marker must be completely visible and segmented from the background. One way to interact with the system is thus to detect if a marker has been obscured by example a finger gesture in front of the camera. Figure 2: Overview of ARToolKit pipeline Advantage with 2D fiducials is that they can be printed not only on a blank piece of paper but also in books and other printed media. This allows 2D printed media to be augmented with related interactive 3D animation. MagicBook is one example of how printed content can be mixed with 3D animation (Billinghurst et al., 2001). This transitional interface spans the Milgram continuum by not only augmenting a physical book with 3D content but also allowing the user to experience the virtual scene in VR mode. Digital content is viewed through handheld video see-through glasses, which resemble classic opera glasses (Henrysson, 2007). 3. Methodology MyviAR Application is developed using android operating system because it is the first software platform and operating system built for open and complete mobile device (Gong and Zhou, 2008). Android platform used in this application is Android 2.3.3 (Gingerbread). 3D models Perodua Myvi is design using Blender software. The AR marker is created using AR Marker Generator and this application is developed in Eclipse IDE using the AndAR library. 3.1 3D Perodua Myvi Car Model Design The software used to design 3D Perodua Myvi model in this paper is Blender. Although Blender is a software that is easier to use than other software like 3D Max, but car modeling is a complex process and requires a lot of technique. There are many problems that have been encountered during 3D Perodua Myvi modeling and require a long time to resolve. Firstly, the 3D car model design cannot be too complex because it will use a large memory space. Android smart phone could not support such a complex 3D model. To solve this problem, developers can use decimate function provided in Blender to reduce the total faces of the model which helps in reducing the memory space. After completing the design, developers must export the car model in.obj format as shown in Fig. 3. The obj class parcel is a library available in AndAR which acts as an advocate for objects, materials and textures. Although the.obj class parcel can support objects in AR, but there are some limitations that need to be addressed during the modeling. Among the restriction are: (Domhan et al., 2010) ISSN: 2289-2265 Page 83

i. Every face must have normals specified ii. The object must be triangulated, this means exactly 3 vertices per face. iii. Basic materials and textures are supported. 3.2 Marker Design Figure 3: Export car model in.obj format. Marker design is an important process to determine whether the AR technology can be demonstrated at the end of the project. Marker must be designed in a square black background. Software used to design the marker is Photoshop. Black rectangle brings the most important attribute in marker. AndAR library can only detect the black squares. Images contained in the black square are not important because the images can be interchanged. Fig. 4 is the marker used in this project. The image in the black square is the Perodua logo. This marker needs to be stored in the.png format. Figure 4: Marker used in this project The next step is to print this marker on a paper and generate it using the online Marker Generator. Marker Generator will detect the marker and store in.pat format. Fig. 5 shows the successful detection and creation of the marker by the Marker Generator. Markers are then imported in.pat format into the Eclipse software to be used in this application. The marker used in MyviAR Application is named as "perodua.pat". ISSN: 2289-2265 Page 84

Figure 5: Marker Generator successfully detect the markers 3.3 User Interaction in MyviAR Application The first step in this process is to import AndAR library into Eclipse software (Sood, 2012). Fig. 6 shows the Java code to create the menu button for the MyviAR Application. The layout of each menu button is in accordance with a written arrangement of Java code. There are four interaction technique in MyviAR Application. They are translate, rotate, scale and take a screen shot function provided for user to run this application. Fig. 7 shows the Java code needed to ensure that this application can detect the marker. Thus, the name written in Java code must be the same with the marker image name. private class ModelLoader extends AsyncTask<Void, Void, Void> { private String modelname2patternname (String modelname) { String patternname = "perodua"; return patternname; Marker Name } Figure 7: Java code to detect the marker ISSN: 2289-2265 Page 85

public boolean oncreateoptionsmenu(menu menu) { menu.add(0, MENU_TRANSLATE, 0, res.gettext(r.string.translate)).seticon(r.drawable.translate); menu.add(0, MENU_ROTATE, 0, res.gettext(r.string.rotate)).seticon(r.drawable.rotate); menu.add(0, MENU_SCALE, 0, res.gettext(r.string.scale)).seticon(r.drawable.scale); menu.add(0, MENU_SCREENSHOT, 0, res.gettext(r.string.take_screenshot)).seticon(r.drawable.screenshoticon); return true; } public boolean onoptionsitemselected(menuitem item) { switch (item.getitemid()) { case MENU_SCALE: mode = MENU_SCALE; return true; case MENU_ROTATE: mode = MENU_ROTATE; return true; case MENU_TRANSLATE: mode = MENU_TRANSLATE; return true; case MENU_SCREENSHOT: new TakeAsyncScreenshot().execute(); return true; } return false; } ; Figure 6: Java code to create the menu button ISSN: 2289-2265 Page 86

4. Result 4.1 Marker Detection The camera need to focus on marker all times so that the 3D Perodua Myvi can displayed. Figure 8 shows the results after the application is successful in detecting the marker. The selected car s color is "orange". Thus, the 3D Perodua Myvi model posted on smartphone screen is orange. Figure 8: 3D orange Perodua Myvi model displayed after successfully detect the marker 4.2 Translate Menu Translate function allow users to switch 3D Perodua Myvi model to any position by simply sliding a finger across the screen of smartphone. Fig. 9 shows the result after the user selected the translate menu. Fig. 10 shows the position of the 3D Perodua Myvi model successfully moved to the bottom after a user slide the finger downwards at screen of smartphone. Figure 9: Translate menu was selected ISSN: 2289-2265 Page 87

4.3 Rotate Menu Figure 10: 3D Perodua Myvi model successfully moved to the bottom Rotate function allow user to rotate the car up to 360 degree to look at every view of the car. Fig. 11 shows the 3D Perodua Myvi model after rotated by sliding a finger on the screen of the smart phone. 4.4 Scale Menu Figure 11: 3D Perodua Myvi model after rotated Scale menu function is to allow users to change the size of the 3D Perodua Myvi model. The size of the model size can be enlarged by sliding the finger to the right on the screen of smartphone. Meanwhile, the size of the model can be reduced by sliding finger left on the screen of smartphones. Slide the finger to the top or bottom will not affect the size of the 3D Perodua Myvi model. Fig.12 shows the original size of the 3D Perodua Myvi model. Fig. 13 shows the 3D model has successfully increased in size by sliding a finger to the right on the screen of smartphone. Fig. 14 shows the model which was successfully reduced in size by sliding a finger to the left on the screen of smartphones. ISSN: 2289-2265 Page 88

Figure 12: Original size of the 3D Perodua Myvi model Figure 13: 3D Perodua Myvi model has successfully increased it sizes Figure 14: 3D Perodua Myvi model has successfully reduced it sizes 4.5 Take a Screenshot Menu After the screenshot is taken, the image is stored in the smartphone s memory card. Figure 15 shows a after the screenshot process, a dialog is displayed to indicate that the screenshot was successful. ISSN: 2289-2265 Page 89

5. Conclusion and Future Work Figure 15: Dialog Screenshot saved! is displayed MyviAR application consists of four functions in the options menu when presenting 3D Perodua Myvi model. The four function is to translate, rotate, and scale the take a screenshot. This advantage allows the user to view 3D Perodua Myvi models from the perspective of front, side and rear with just sliding a finger. Users also can enlarge the 3D Perodua Myvi model to see the design more clearly. Furthermore, this application contains a list of seven color of Perodua Myvi to facilitate users. Users can view the 3D Perodua Myvi model with different colors according to their desire. Fig. 16 shows the list of Perodua Myvi colors available in MyviAR Application. Figure 16: A list of seven color of Perodua Myvi For ease of use, navigation menu when presenting 3D Perodua Myvi model should be added. Researchers can add the "next" and the "previous" button so that the user can see the 3D Perodua Myvi model with different colors by a click away. Fig. 17 shows an example of an interface that contains the function "next" and "previous" and can be used by other researchers if they want to develop this application in the future. ISSN: 2289-2265 Page 90

Figure 17: Example of an interface that contains the function "next" and "previous" References [1] ABAWI, D. F., BIENWALD, J. & DORNER, R. Accuracy in optical tracking with fiducial markers: an accuracy function for ARToolKit. Mixed and Augmented Reality, 2004. ISMAR 2004. Third IEEE and ACM International Symposium on, 2-5 Nov. 2004 2004. 260-261. [2] AZUMA, R., BAILLOT, Y., BEHRINGER, R., FEINER, S., JULIER, S. & MACINTYRE, B. 2001. Recent advances in augmented reality. Computer Graphics and Applications, IEEE, 21, 34-47. [3] BAGAL, N. S. & KALE, N. Android open-source operating System for mobile devices. [4] BILLINGHURST, M., KATO, H. & POUPYREV, I. 2001. The magicbook-moving seamlessly between reality and virtuality. Computer Graphics and Applications, IEEE, 21, 6-8. [5] DOMHAN, T., TIT07INA, K. & DER DUALEN HOCHSCHULE, G. 2010. Augmented Reality on Android Smartphones. Studiengangs Informationstechni. Dualen Hochschule Baden- Württemberg Stuttgart. [6] GONG, L. & ZHOU, C. 2008. Development and Research of Mobile Termination Application Based on Android [J]. Computer and modernization, 8, 85-88. [7] HENRYSSON, A. 2007. Bringing augmented reality to mobile phones. [8] JUIN, Y. C. 2012. Katalog Kereta Berasaskan Realiti Tambahan Dalam Persikitaran Android. Universiti Kebangsaan Malaysia Bangi.. [9] MILGRAM, P., TAKEMURA, H., UTSUMI, A. & KISHINO, F. Augmented reality: A class of displays on the reality-virtuality continuum. Photonics for Industrial Applications, 1995. International Society for Optics and Photonics, 282-292. [10] RAMLEE, J. 2011. Perodua allocates RM613.2m for Capex. New Straits Times Press. [11] SCHALL, G., NEWMAN, J. & SCHMALSTIEG, D. Rapid and accurate deployment of fiducial markers for augmented reality. Proc. 10 th Computer Vision Winter Workshop (CVWW 2005). http://www. icg. tu-graz. ac. at/pub/pubobjects/schall05cvww [2005, 2005. [12] SILTANEN, S. 2012. Theory and applications of marker-based augmented reality. [13] SOOD, R. 2012. Pro Android augmented reality, Apress. [14] WAGNER, D. & SCHMALSTIEG, D. 2009. Making augmented reality practical on mobile phones, part 1. Computer Graphics and Applications, IEEE, 29, 12-15. ISSN: 2289-2265 Page 91