Gesture Based Wireless PC Control with Gyroscope and Accelerometer

Transcription

1 Gesture Based Wireless PC Control with Gyroscope and Accelerometer Sabrina Tarannum 1, Mohammed Saifuddin Munna, Mohammad Ariful Islam Bhuyan 3 Department of Computer Science & Engineering 1, Department of Electrical and Electronic Engineering, 3 Premier University, Chittagong Chittagong- 403, Bangladesh. saaabrin@gmail.com 1, munna.puc@gmail.com, arif.ajtfs@yahoo.com 3 Abstract This paper proposed a design to control a PC with the virtual PC mouse by hand gesture. There are different types of methodology used to control virtual PC mouse i.e. kinetic sensor, webcam, optical sensor etc. But in our proposed design author used just an IMU, which is an inertial measurement unit. IMU combined with accelerometer and gyroscope. Firstly accelerometer and gyroscope track different gesture of the human hand. Then instruction sends over Bluetooth protocol. Inertial displacement data scaled for a frame and helps to move the pointer in different vector position. A JAVA programmed application is used in the Computer to perform the operation of Mouse. The total system and results are discussed later. Keywords-virtual; accelerometer; gyroscope; gesture; inertial measurement unit; JAVA programed ; bluetooth I. INTRODUCTION The use of computer makes human life easier. We cannot imagine our modern life without computer and its applications. In human computer interaction mouse is still commonly used input device. Now to control a PC mouse improved day by day. Many researchers used different methods to control PC mouse wirelessly. With rapid advancement in the field of human computer interaction, it has become possible to gain easy access and control of computer applications using gestures. The D mouse with three degrees of freedom can t satisfy their demands. They hope the mouse can control the rotations around the axes besides the movements along the axes. Our proposed design based on accelerometer and gyroscope technology. We designed an artificial algorithm which helps to track the motion of hand and its gesture. Wireless device is interfaced over Bluetooth protocol. On the other hand the receiver section has a JAVA programmed startup application. Which is used to auto start the program when the computer startup. Then JAVA program detect the data from sender & calculate it for mouse pointer position change and clicking option. Moreover our proposed system is very cheap and user friendly. II. RELATED WORKS The hand and fingertips detection have been studied for several decades, for which methods designed for gesture recognition without any webcam and complexity. In 008 PC mouse operations using mouth open/close motions and head tilting is designed where a head sensor system is used. From the sensor, they collect the data from head movement angel & from this the roll & pitch is calculated. And a distance sensor is also placed in front of the mouth to calculate the mouse position.[3] Another design for this types of device use gyro opto sensors for the operation where Optical IR sensor used to calculate the eye blinking & gyro calculate the body movement. [4] Another work on this particular field works with image processing with the help of kinetic sensor. Here the movement of hand calculated for displacement & click also. [5] III. SYSTEM OVERVIEW Our total system designed with two parts. One is remote device which is designed, and another one is PC processing unit. Remote part captures the hand gesture and motion. After that data processed by microcontroller then sends to PC unit over Bluetooth. PC unit have processing unit and then make decision to move the pointer in desire position according to hand gesture. IV. METHODOLOGY The whole system divided in two sections which are described below A. Block Diagram In this Proposed design there are two different sections first one is transmitter section and Second one is receiver section.

2 connection setup. Serial port Bluetooth module is fully qualified Bluetooth V.0+EDR (Enhanced Data Rate) 3Mbps Modulation with complete.4ghz radio transceiver and baseband. It uses CSR Bluecore 04-External single chip Bluetooth system with CMOS technology and with AFH(Adaptive Frequency Hopping Feature). It has the footprint as small as 1.7mmx7mm. Hope it will simplify your overall design/development cycle.[1] Figure 1. Block Diagram of Wireless Transmitter Section. From figure 1. there four parts in transmeter section. Power suply unit Sensor Unit Processing Unit Sender Unit Figure 3. HC-05 module In power suply unit it s a simple DC power suply of 5 volt. In the sensor unit author used differnet types of sensor for this proposed device. Accelerometer Gyroscope Here accelerometer is used as right click & left click option. And by using a single gyroscope this work is done. b) Arduino UNO R3 : It contains Atmega AVR 38p micro controller. It has 14 digital Output/Input pins (of which 6 can be used as PWM signals), 6 analog inputs, a 16 MHz crystal oscillator. Operating voltage of the micro controller is 5V. Contains 3 KB of flash memory enough to store the data for this particular project.[1] In order to have proper mechanism and response from the robot, proper readings had to be taken for it. In this I have used 3 sensors: Figure. Block Diagram of Receiving Section. On the other side the reciever unit have two differnt sections Receiver unit Bulethoot Processing unit This system use a JAVA based program as processiing unit of the resciever section. B. Hardware Unit HC-05 Bluetooth Module Arduino UNO R3 Gyroscope Accelerometer Flex sensor a) HC-05 Module: HC-05 module is an easy to use Bluetooth SPP (Serial Port Protocol) module, designed for transparent wireless serial Figure 4. Ardiuno Uno c) ADXL335 : It is a 3- axis accelerometer sensor, which can measures the forces applied on to the sensor in all the 3 directions X, Y and

3 Z axis. Further the raw data from the sensors are converted into acceleration by using some complicated equations. The advantage of the accelerometer was that the values do not change unless there is a change in position. But the problem with the accelerometer was that it contained high level of noise which makes the values inaccurate. So, to make these values accurate Gyroscope sensor was used.[1] resistance ohm and at 180 degree bend resistance 000 ohms.[1] This resistance was calibrated and converted into angles and send the data of angel to the controller. C. Calculaton of angel by gyrosocope & acclerometer For calculating the amount of hand movement for cursor movement, some calculation has to be done from the raw data collected from the sensors. Let, acceleration component denote as x a,y a and z a and gyro component denote as x g, y g and z g. If R vector on the XYZ axes. Please notice the following relation: R R x Ry Rz Since the tilt of the X-axis actually shows rotation around the Y-axis, and the angling of the Y-axis shows (negative) rotation around the X-axis. Figure 5. ADXL335 sensor The rotation around the X-axis (φ) d) L3G400D: It is a 3- axis Gyroscope sensor which can measure the degree of rotation in all the 3 axis in form of alpha, beta and gamma. The values provided by the Gyroscope are very accurate but values do not remain static and tend to drift to the position Zero. To make the readings accurate as well as static both the values from the Accelerometer and Gyroscope were combined by using Kalman filters.[1] φ tan 1 The rotation around the Y-axis (ρ) ρ tan 1 y a x a + z a x a y a + z a If the Z-axis is aligned along the gravitational acceleration vector, then it is impossible to compute rotation around the Z axis from the accelerometer. That s why need gyro values to find out Z rotational angle. GyroXgyroX/131.0 rate GyroY(gyroY/131.0) rate Where 131 is constant. The rotational angle along with Z-axis can be define as gyro_ angle _ z (gyroz rate dt) lastz angle kalanglexaccx angle, gyrox rate ) kalangleyaccy angle, gyroy rate ) Figure 6. Gyroscope sensor All of these equations are generalized form and shows the calculation methodology. To perform actual Kalman calculation library function needed. [] e) Flex Sensor : Flex is basically a strip of carbon material having metal pads inside it. As the sensor is flexed, the resistance across the sensor increases. The resistance of the flex sensor changes when the metal pads are on the outside of the bend Straight (un-flexed) resistance: 9000 ohm, 90 degree bend

4 Figure 8. Processing Unit of Transmitting Section. From the above figure 7, when the power of our proposed sytem is turn on then it starts to collect data from the sensors. Then the raw data is come to the ardiuno uno for proper calcuation and filtering. It calculate the hole plane a 3D plane of Then it calculate the current positing of mouse and then calculate the position change and filter the data for our desire level. After calculating the position change dta it sends it to the reciver via bluetooth. Figure 3. Geometry of angle calculation using accelerometer and gyrometer.[] D. Schemetic diagram of transmiter section: On the Other hand two different accelerometer sensors are palced into tow finger position. One for right click and the second one is for left click. When used click his nd finger it will send a 1 bit data to the controller & for middle finger it will produce a different bit. And when tow fibgers are moved at a same time it will work as double click. In proposed system, the transmitting Controller creats differnts signal for every command. And using the HC-05 Module bluetooth Module it will send s to our reciever section. b) Reciever Secsion: Figure 9. Processing Unit of Receiving Section. In the this part from the flow chart shows in figure 8, we first recive a data via computer Bluetooth port or external bluethhot reciver. At first our designed JAVA language program set to startup manu for auto start the program when the computer starts. This JAVA program used ROBOT class for mouse pointer movement. E. Algorithm Figure 7. Schemetic Diagram for Transmiter Section a) Transmitter section: When it takes a data bit via bluetooth then processing section stars to work. Calculate the change of position, instruct the pointer to move to the desire position. In case of Click r cherecter data from sender is processed to work as right click & l cherecter from sender work as left click. When the processor find two l data with in 0.s it will work as double click.

5 V. RESULTS [5] Szilvia Szeghalmy, Marianna Zichar,Attila Fazekas, Gesture-based computer mouse using Kinect sensor, 5th IEEE International Conference on Cognitive Infocommunications Figure 10. Designed transmission section. Figure 9. show our designed hardware unit. Where accelerometer & a gyroscope is connected with the transmiter bluetooth. O n the other hand we collect some data from practical operation using our proposed design. We did it for different distances. bellow data table show the value of results. Observation no. Distance from receiver (Meter) TABLE 1: RESULT DATA Single click Time required Double Click Cursor movement 1 8 1s 1s (3-4)s 7.5s 1s (-3)s s 1s s 4 5 No delay 0.5s 1s 5 3 No delay No delay 6 1 No delay No delay (0.5 1)s No delay REFERENCES [1] Shamsheer Verma Hand Gestures Remote Controlled Robotic Arm, Advance in Electronic and Electric Engineering. ISSN , Volume 3, Number 5 (013), pp [] Ariful Islam Bhuyan1, Tuton Chandra Mallick, Gyro-Accelerometer based control of a robotic Arm using AVR Microcontroller, 9th International Forum on Strategic Technology (IFOST), 014. [3] Nobuaki NAKAZAWA, Naoto HASHIMOTO and Kou YAMADA, Welfare-Support Interface for PC Mouse Operations Using Mouth Open/Close Motions and Head Tilting, Industrial Electronics, 008. IECON th Annual Conference of IEEE [4] Soochan Kim,Minje Park,Sasiporn Anumas, Jaeha Yoo, Head Mouse System based on Gyro- and Opto- Sensors, 3rd International Conference on Biomedical Engineering and Informatics (BMEI 010)