Haptic control and feedback in mobile gaming. Tommi Urtti

Transcription

1 Haptic control and feedback in mobile gaming Tommi Urtti University of Tampere Department of Computer Sciences Haptics in mobile contexts -course thesis December 2008

2 Table of Contents Haptic control and feedback in mobile gaming... 1 Introduction... 3 Pervasive elements... 4 The accelerometer... 5 Multi-touch screens in gaming... 8 Tactile feedback... 9 Conclusion References Keywords: Pervasive gaming, haptics, gaming, accelerometer, multi-touch

3 Introduction Gaming is all about immersion: a good game is one that takes you into the game and doesn t let you go, the type of game that you play just one round more. Many successfully designed features of the game are needed to achieve the best possible immersion. These include graphics, sound, plot, characters and challenge.[5,6,7] One element that has not been fully integrated until late has been the use haptic feedback and use of haptics as input. Including haptics has been shown by Ramsamy et al. [14] to increase immersion considerably. There are other cases in which haptics has been integrated as only an inconsistent and irrelevant vibration of the game controller. In these cases, most of them console or PC games, many players find the addition annoying instead of positively contributing to the experience. In the past few years many interesting innovations have electrified the usage of haptics in games. Nintendo Wii was marketed before announcement as a revolution of gaming instead of an evolution like its competitors the Xbox360 and the PS3. Wii s graphics are not on par with the other consoles, but its motion-sensing haptic game controller has made it a success story worldwide, outselling [15] both Microsoft s and Sony s consoles. Apple s iphone was introduced in June The much-hyped smartphone has many technical capabilities that profile it as a sound mobile gaming platform. These include a 3,5 inch multi-touch display, 3D-graphics processing unit from PowerVR and an accelerometer. Game developers have welcomed the accelerometer and multi-touch display. We introduce these features through example games. Sony Playstation s DualShock controller sparked interest in vibrating controllers in the 1990 s. Afterwards there has been little progress in designing and giving players haptic feedback in attempts to achieve a more immersive experience. We ll take a look at few innovations in this area, particularly in the field of pervasive gaming, where haptics is expected to play a bigger role than with mainstream gaming.

4 Pervasive elements Pervasive elements are designed to bridge real life objects into virtual game environments. We decided to include this part in the survey, since they introduce interesting ideas to gaming in general and mobile gaming in particular. Ullmer and Ishii [17] have in their work presented two levels of haptic feedback. In tangible interfaces the first one is the representation level. Immediately by picking up a tangible object like a baseball bat or a real life PDA you get haptic sensations from the device, what is its form and functions. The other level is the interaction level, this the haptic feedback given to the player when the physical objects and digital information interact in pervasive games. For instance when you hit an image of a tennis ball with an actual tennis racket (or a controller) the racket vibrates. Martin Faust has accomplished some interesting work with pervasive elements in games. Doom 3, a first-person shooter game, has the typical method of guiding the player with in-game s and a heads-up-display. Faust modified the game [8] in a way that instead of reading the s on the on-screen virtual PDA, they were pushed to an actual PDA device. Haptic feedback of the game was improved on the representation level, since the player could hold their real PDA in their hand and read the s on it, instead of watching an on-screen image of a PDA. The new version of Airkanoid [8] is another work by Faust. Airkanoid is an old game from 1985 where the player controls a paddle with which he reflects a ball around the screen to break a number of bricks. Once the bricks are broken the player reaches the next level. A new version of Airkanoid in 2006 took the game concept as it was, but made a few changes. The paddle movement would not be vertically limited as it had been before (paddle on the bottom of the screen, horizontal movement for the width of the screen). The big invention however was to replace the virtual paddle with actual physical objects called Airbats. The Airbats in the game are optically detected, so they can essentially be anything, even a rolled up paper. The game is projected on to a screen. The image has been altered so that the game elements are added on top of the video stream of the players. When the players position the Airbat so that the ball hits it, the ball is reflected in the game (see figure 1).

5 Figure 1: Haptic Airkanoid. In Haptic Airkanoid Faust and Yoo [10] further improved the concept of Airbats. By adding two vibrating elements into the Airbats, a stronger bond could be created between the virtual and physical world. When the image of the physical Airbat hits the image of the ball, the Airbat vibrates like the collision had actually happened. The participants described the sensation as not very realistic, but all of them agreed that the game was more fun with the haptic feedback than without. The accelerometer The accelerometer (or motion sensor) is not a new invention, but lately it has appeared more commonly in consumer devices. Hard drives have integrated the motion sensor to prevent damage from sudden movement. This is especially useful in laptops. Mobile phones, including the Nokia N95 and Apple iphone, are equipped with accelerometers. Other mobile phones equipped with this feature include models from Samsung, Motorola and Nokia. In terms of software distribution the iphone has an advantage in comparison to competitors. Games and other software can be purchased exclusively from the App Store. App Store has proven a success; Vodafone and other operators are now developing similar services for other phone models. iphone is the main platform for which we surveyed accelerometer games. iphone s accelerometer composes of a silicon mass, a set of silicon springs, and an electrical current. Motion is sensed by registering fluctuations in the electrical current when the silicon mass moves. The silicon springs are in place to lead the current

6 through the mass. Fluctuations are registered and software developers may use the data as events. [2] Figure 2: Accelerometer demonstration [Source: Apple.com]. The accelerometer in the iphone is used by many of the games available on App Store; one of these games is Labyrinth. The game s idea is rather simple, to guide a small ball through a maze with holes for the ball to fall through. In the original game balancing the board was done via 2 wheels that rotate the axis of the planes. This is type of game that would be practically impossible to implement with good immersion using the typical keypad for control. Labyrinth on the iphone uses the accelerometer to measure the angle of the phone. When the phone is slanted the ball rolls with fairly realistic physics in the correct direction with speed relative to the angle.

7 Figure 3: iphone game Labyrinth. Other available games thatt use the accelerometer include racing, snowboarding, fishing and flying (a bi-plane). Generally the player moves the phone around using it as they would a steering wheel or a fishing rod. Skeptics have questioned the actual immersion of accelerometer games believing it to be something that wears off quicker than a captivating plot or a well-designed learning curve. Asier Arranz developed software called NiiMe [3] (Nokia2MovingExperience) in The software is an innovative design that enables the use of Nokia N95 and other Nokia models equipped with the accelerometer to be used as a control peripheral of a PC. Software installed in the Symbian S60 phone sends location data (x,y,z-coordinates) via Bluetooth to a nearby PC that has been paired with the phone. The phone can then be used as a mouse or a game controller. Game controller usage has been demoed with games that use the phone as a drumstick, steering wheel, airplane flight control and playing Super Mario by actually moving your feet to run and jumping to jump. The last one I believe is done by wearing the phone in the player s pocket and registering a slight horizontal coordinate change for running and a clearer vertical change for jumping. In my opinion this represent a clever usage of

8 technology together, taking the accelerometer of the Nokia phone and using to control a gaming platform that can run more advanced games. It is early to say if the accelerometer will become a common feature of smartphones aiming the gaming market. Nevertheless it gives developers the possibility of designing a completely different game experience compared to the traditional keypad. Multi-touch screens in gaming Multi-touch has been one of the most discussed features of display techniques since Jefferson Han s (NYU's Courant Institute of Mathematical Sciences) impressive demonstrations in the TED (Technology, Entertainment, Design) conference of In the following years a similar concept was implemented with different techniques into the Apple iphone and Microsoft Surface. Multi-touch will also be supported in the upcoming Windows 7 operating system, scheduled for launch late 2009 or early Multi-touch makes gestures a possible method for input in a user interface. For example the iphone has the flick, tap, pinch and stretch gestures [1]. The gestures are used in an attempt to create more intuitive interfaces with less buttons, scales and menus. By reducing the number of physical buttons in mobile devices the frame can fit a larger display. A larger display providing visual feedback has been found to increase game immersion. Previous generations of touch screens had error situations when the touch screen was used too quickly. Multi-touch has been designed for simultaneous touches and thus will register touches with better accuracy being less prone to errors. Disadvantages of touch screens quickly become obvious when multiplayer games are played on a single screen. Fingers block the view of the display. AirHockey is one of the games that use multi-touch on the iphone. Two players can play the game either on one phone or over the network. AirHockey is a game where you control a circular tool with which you hit the puck trying to score a goal in the other player s end. The game is fast-paced and requires constant attention. As a result typically you will have both players controlling their sticks all the time. This game could not have been implemented on a touch screen without the multi-touch

9 capabilities of the display. A normal touch screen would only register the one player s movements. A different type of example usage of multi-touch is Simcity, a classic city construction simulation game on the PC. Simcity on the iphone uses the pinch and stretch gestures to zoom in and out. Previously it has been published for the Nintendo DS and the Windows Mobile platform. Once this game is published it could be well be used for comparing gaming with gestures or a stylus on mobile platforms. Tactile feedback Typically haptics are used in games for feedback instead of an input. Most mobile phones have the vibration function that is used in games for simple feedback. Problem with the typical vibration function of a mobile phone or a game controller is that it is too ambiguous: it doesn t give specific information to the player of what has happened. In attempts to significantly improve the usefulness of tactile feedback innovative engineering is needed. Well designed haptic feedback was proven by Jiang et al to reduce error rates and thus improve performance [11]. TN Games was founded by a medical surgeon who invented a new method of using old aircompression techniques. The TN Games 3 rd Space Vest has 8 pockets that inflate rapidly creating the sensation of a tap. Depending on a game this will be getting shot or hit. Giving the player a hint where the attacks are coming from enables a quick response and a more immersive experience [16].

10 Figure 4: TN Games 3 rd Space Vest. There are many different ways of giving haptic feedback to the player. Techniques include usage of ultrasound or deep bass sounds. If a company wants to concentrate on making a product as mobile as possible sound and air inflation devices are heavyset. Also in order to achieve the most mobile experience, cables must get rid of. 3 rd Space Vest has a USB cable running to the PC and an air tube going to the external compressor, which has an AC cord. With this setup moving is going to be limited and the user has to weigh the value of the haptic feedback. Scott Lowe, of IGN Gear, was one of many that was impressed by the TN Games device, but would not wear it regularly. [13] Conclusion In this paper we have given an overview of some of the recent developments in the field of enhancing gaming through the use of haptics as either an input or output. As future work it would be interesting to do user studies comparing using haptics as input against keypads or joystics.

11 References 1. Apple Inc., iphone multitouch specification sheets, ( ) 2. Apple Inc., iphone accelerometer specification sheets, ( ) 3. ARRANZ, A. ( ) 4. BROWN, E. and CAIRNS, P., A grounded investigation of game immersion, CHI 04: CHI '04 extended abstracts on Human factors in computing systems, 2004, ACM pp CHANG, D. "Haptics: Gaming's New Sensation," Computer, vol. 35, no. 8, pp , Aug., CHENG, K. and CAIRNS, P.A., Behaviour, realism and immersion in games, CHI'05: CHI '05 extended abstracts on Human factors in computing systems, 2005, ACM pp ERMI, L. and MÄYRÄ, F., Fundamental components of the gameplay experience: Analysing immersion. Proceedings of DiGRA 2005 Conference: Changing Views - Worlds in Play FAUST M., Airkanoid, ( ) 9. FAUST M.: Doom 3 PDA Mod, ( ) 10. FAUST M., YOO Y.-H.: Haptic Feedback in Pervasive Games. In Proceedings of the Third International Workshop on Pervasive Gaming Applications - PerGames JIANG, L. et al.: Reducing error rates with lowcost haptic feedback in virtual reality-based training applications. In: Proceedings world HAPTICS, Pisa, Italy (2005) KIM, S.-Y.: Interactive Racing Game with Graphic and Haptic Feedback, p.69-77, LNCS Volume 4813/ LOWE, S. 3rd Space Gaming Vest Review, ( ) 14. NEXGENWARS, ( ) 15. RAMSAMY, P. et al: Using Haptics to Improve Immersion in Virtual Environments. In Computational Science ICCS 2006, Springer 2006, pp. 16. TN GAMES ( ) 17. ULLMER, B., ISHII, H.: Emerging frameworks for tangible user interfaces. In Caroll, J.M., ed.: Human-Computer Interaction in the New Millenium. Addison-Wesley (2001) WANG, Q. et al: A Haptic Memory Game using the STRESS Tactile Display CHI 2006, April 22 27,