Human Activities Recognition in Android Smartphone Using Support Vector Machine

Transcription

1 2016 7th International Conference on Intelligent Systems, Modelling and Simulation Human Activities Recognition in Android Smartphone Using Support Vector Machine Duc Ngoc Tran Computer Engineer Faculty University of Information Technology Ho Chi Minh, Vietnam Duy Dinh Phan Computer Engineer Faculty University of Information Technology Ho Chi Minh, Vietnam Abstract In this study, we designed and constructed a system to identify human actions using integrated sensors in smartphones. There are six actions that are selected for recognition include: walking, standing, sitting, lying down, up the stairs, down the stairs. In this system, Support Vector Machine (SVM) is used to classify and identify action. Collected data from sensors are analyzed for the classification model - the model file. The classification models are optimized to bring the best results for the identified human activity. After forming the classify model, the model will be integrated into the system to identify the human activities. Human activities recognition system is written on Windows and Android platforms and operate in real time. The accuracy of the system depends on selected features and the quality of the training model. On the Android system running on smartphone with 248 features achieve 89.59% accurate rate. Keywords - SVM, human activities recognition, Android I. INTRODUCTION In the world, the human activities recognition, which use sensors to recognize human actions, have been studied for a long time to produce the more simple system with high precision [1]. However, there is very limited number of project that investigate a human activity recognition system built right on the smartphone. A great advantage of this integrated system is the real time and full time supervision. The human activities recognition built in smart phone promises to open up a new direction not only in monitoring and health care but also in other fields. In fact, the proportion of the population aged 65 and over in the world is projected to grow from an estimated 8 percent of the world s population in 2010 to nearly 16 percent in 2050 [2]. This increase will put the health care institutions under very large pressure, specifically considering the fact the health care cost per capita for persons over 65 years are three to five times greater the under 65 [3]. Thus, medical institutions are actively seeking cost-cutting solutions. The patients who need the medical assistant can be monitored remotely. Through the identification of human actions, the monitoring health care application will provide medical staff with more crucial information of particular patients for giving warnings for accidents such as falling down [4]. This process can be done automatically to reduce the workload of medical institutions. In order increase feasibility of such application, this study focus on some particular activities of human activities recognition include walking, up, down, sitting, standing and laying. Nowadays, smartphone is going to get more popular in the world over the next five years. According to Ericsson mobility report, there will be a massive jump from the 2.6 billion smartphone users recorded in 2014 to 6.1 billion by 2020 [5]. The number of smart devices, which always beside everyone, is quite huge. Moreover, it is built with many sensors to increase the interaction ability for user includes acceleration and gyroscope sensor. Thus, the idea of utilization of these sensors to make a smartphone application for human activities recognition become more realistic. In detail, acceleration sensor measures acceleration in three orthogonal axes. All of objects in the Earth are affected by the gravity. The linear acceleration measures the acceleration effect of the device movement, excluding the effect of Earth's gravity on the device. The gyroscope uses Earth s gravity to help determine orientation of smartphone. The combination of parameters which are collected from these sensors allow to determine the status and the change of physical movement of smartphone in the space. The large collected data provide many important data to recognize the human physical activities. II. RELATED WORKS A. Support Vector Machines In the last decades, there were several machine learning methods that can use for classifier and recognition of human physical activities including Naïve Bayes, Support Vector Machines (SVMs), Threshold based and Markov chain [6]. Although there is not any study that can find out the best method for human physical activities classification, but SVMs have been successfully widely used in many research related to handwriting recognition and speech recognition. Therefore, in this study, SVMs method will be used to classify and recognize human activities. In order to find out the best hyperplane for data classification, SVMs search the hyperplane which has the largest margin. Figure 1 shows both two hyperplanes can be divided in two class. However, figure 1.b shows the larger margin between two classes than figure 1.a. The larger margin will help the classification in next modules easier and avoid mistakes as much as possible. Thus, in SVMs /16 $ IEEE DOI /ISMS

2 classification process, the system algorithm will search to find the hyperplane that have the largest margin. Figure 1 SVMs classification In particular, Anguita introduced the concept of Hardware-Friendly SVM [7]. The fixed point arithmetic is exploited in the feed-forward phase of SVM classifier. This model is extended for multiclass classification. Because this research achieved an average accuracy of 89% with small among of memory, it has good advantage when used in limited resources hardware devices like smartphone. The results of this study will be used for the evaluation of human activities recognition developed in this study. Duy Tam Gilles Huynh did a study on the application of sensors worn on the body to recognize the actions of humans [8]. This study not only focuses on the identification of human activities in the short time but also focus on the identification of human activities in the long period of time. In Android, Google has been introduced a set of API (Application Programming Interface), which allow the developer to connect Google services to their Android phone for receiving the human activity recognition results [9]. Google API can recognize six type of activities include in vehicle, on bicycle, on foot, running, still and walking. However, this system required the connection to Google server in order to send the requests and receive the results. In ios, Apple has released ios Health on ios 8 [10]. The system used GPS to locate service and motion sensors to recognize human activities such as walking, running, biking, thereby calculating the energy consumption of a person in a day. The recognition system focused on the time the user performs the action through two states: active or inactive, this interval should be greater than thirty minutes per day. Recently, research on identifying the action is still being implemented and applied on many topics such as: identifying daily activities [11], the health-care applications, the unsupervised learning method of human activities, etc. In this study, a new method using SVM for human activities recognition is introduced. The practical system is designed and implemented in Android OS smartphone. III. IMPLEMENTATION Human Activities Recognition System are formed from many functional blocks. Each block performs a different task for each training process and identifying actions. The human activities recognition system consists of four main functional modules include: - Data acquisition and data processing module - Feature extraction module - Training module. - Human activities recognition module on smartphone. A. Data acquisition & data processing module This module collects data and processes signal from the smartphone's sensors module when performing the human activities. This block consists of two main components include the controller (as a software running on a Windows computer or on a smart phone) and a data recording device (smartphone). In order to execute the data acquisition process, this module is implemented in two steps. Step 1: establishing a connection between the controller and the data recording device. Step 2: Turn the control signal start gathering process. UDP Datagram TCP Request TCP Connection Recording device Controller Figure 2 Connection interface between recording device and controller B. Feature Extraction Module The raw data collected from the sensor cannot be used directly for SVMs methods. Thus, a module, which conducts calculation for feature extraction and converting raw data to training samples, will be implemented. Those samples have the proper structure according to the requirement of machine learning algorithm. Data are collected from 3 types of sensor: acceleration sensor, gyro sensor and accelerometer sensor linearity. Each sensor returns three values corresponding to threedimensional x, y, z. The sensor runs at 50Hz (collecting 50 values per second) to store raw data in units of sample. Each sample includes 128 values corresponding to each sample time record of 2.56 seconds. Nine arrays of original raw data will be undergone to conversion functions to form the array of data to process features extraction calculations. The conversion functions include: diff: calculate the difference between the value of two consecutive values in array 65

3 mag: calculate the value of the magnitude of the three variables x, y, z fft: convert signal from the time domain to the frequency domain A total of 33 arrays of data are formed after previous conversion functions. The features extraction process will calculate some parameters which are defined based on a set of values (also called the window) on the data stream. There are three parameters need to be evaluated and selected as below [8]: - The length of the window (the number of values in a set of calculations): With basic activities like walking, standing, sitting, lying, the recording time is often 2.56 seconds corresponding to each window which has 128 values [7]. - The displacement of consecutive windows (the windows may overlap to achieve coverage on the entire amount of raw data recorded): A 50% overlap of data windows will be apply to the data [7]. This means the next window will regain 50% of the data of the previous window. - The features parameters calculated on data: Some features are used to calculated is listed in table I. TABLE I. SOME EXTRACTED FEATURES FROM RAW DATA Features Description Mean Average of the array Std Standard deviation Mad Average of std Max Maximum Min Minimum SMA Energy Energy of signal Iqr Entropy These features will be calculated in from the time domain and the frequency domain. There are two version of this system include MATLAB version for testing in PC and Android version for implementation in smartphone. In MATLAB, total 561 features has been calculated. In other side, Android version has 248 features. C. Trainning Module Training module convert extracted features to the recognition model, which will be used as a template for activity recognition. The specific data to be arranged in a special format, this format to comply with regulations on data formats of classify of SVMs. The data will be processed with the library toolkit SVMlib [12]. The results show that the models which support the activities recognition. Results of recognition will be analyzed, compared to find out the fault location as well as features that are calculated from those faults in order to adjust the calculation to achieve better recognition model. After assessing the results, any recognition model that has over 80% of accuracy will be selected to use in the recognition module on the smartphone. D. Recognition Module Because the system requires real time operation, the recognition module is processed in a short time of 3-5 seconds. In particular, this period of time includes the time of writing data and the time of recognition for activity. Writing data module and processing data from sensors module will record data for 2-3 seconds (128 values from each sensor). The amount of data will be processed through the features extraction module. These features are calculated on time domain over time. Totally, 248 features were calculated. Then, these features will be recognized with the training model, which obtain from training module. Recognition results will be sent to the smartphone user. Due to the requirement of continuous activity recognition, the calculation should be done in parallel with the process of writing data, which is used for the next recognition. IV. RESULTS A. Testing Environment The experiment were carried out on 10 volunteers aged 11 to 26, who have normal health. Each volunteer will perform six basic activities. The process of data collection is controlled by a computer program, data will be named separately to manage, store. Data will be accompanied by information on the action name, the name of volunteers and time data logging. Then the data will be randomized into two data sets: 70% for the training, 30% for the inspection process. The training will be done by SVMs. Xperia Z1 is used in this test. Smartphones contain three essential sensor including accelerometer sensor, linear acceleration and gyroscope. The sensor data recorded at 50Hz, suitable for recording data on human activities. To implement the process of identification, an application on smartphones running Android OS has been developed. The process begins by identifying the collection of raw data from the sensor. These data will be divided into small data samples, each sample is a sequence of 128 values corresponding to the time of collection of 2.56 seconds. Then the sample data will be sent to the characteristic calculation. Finally these characteristics will be included in the identification of SVMs to identify the action. The entire process of identification is shown in Fig 3. Figure 3 Human activities recognition procedure 66

4 B. Data Collection The raw data was collected from 10 volunteers aged from 11 to 26. All of them have normal health status. The phone place in their pocket. The data is collected from samples as in Table II. TABLE II. SAMPLES COLLECTED FROM VOLUNTEERS Type of activities Number of samples Sitting 3541 Standing 4252 Walking 1635 Upstair 360 Downstair 260 Lying 891 D. Recognition on Smartphone The Human Activity Recognition System on smartphone is developed on the Android platform. The program will collect data from sensors, perform calculations and make comparisons with the model which has been chosen before. The result is sent to the server or displayed on the phone screen as Fig 4. C. Training and Evaluation In order to compare with the previous research from Anguita, the first version of this study is processed with Human Activity Recognition Using Smartphones Data Set which is used as the testing data for Anguita s research. Table III shows the recognition of 2752/2957 samples (93.38%), which means quite higher the previous study result (89%). On other hand, the second version is tested with 248 features, which reduce from 561 features from the first version, to the decease the volume of calculation and achieve the real time recognition ability for smartphone implementation. In general, the system can recognize correctly human activities with high detection rates in above 89% as shown in Table IV. However, lying activity has low recognition rate and be perceived to act sitting. The reason is the similar status of smartphone when the user do the sitting and lying activity. TABLE III. RESULTS OF THE RECOGNITION RUNNING WITH ANGUITA S DATA SET Walking Up Down Sitt Standing Lying % ing Walking Up Down Sitting Standing Lying Total TABLE IV. RESULTS OF THE TRAINING PROCESS AND RECOGNITION WITH 248 FEATURES Walking Up Down Sitt Standing Lying % ing Walking Up Down Sitting Standing Lying Total Figure 4. Results in controlled software V. CONCLUTION AND FUTURE WORK In this paper, the research team has developed a complete system to recognize the human activity. The completed system included data acquisition systems, features extraction, data processing, training and human activity recognition. This system can be applied in many fields of practice especially health care field. One particular application as identification of patient falls, the index measuring applications advocacy etc. However, the system still has some certain restrictions. Percentage of recognition is low in some action. In the future, there should be more research to improve the performance and increase the detection capabilities of the system. ACKNOWLEDGMENT The research is funded by Computer Engineering Faculty in project C2014-5, University of Information Technology Vietnam National University. REFERENCES [1] T. Starner, B. Rhodes, J. Weaver, and A. Pentland, Everyday-use Wearable Computers [2] N. I. on Aging, Humanity s Aging, National Institute on Aging, 26- Mar [Online]. Available: [Accessed: 08-Jan-2016]. [3] S. Jacobzone and H. Oxley, Ageing and Health Care Costs, Internationale Politik und Gesellschaft Online, [Online]. Available: [Accessed: 08-Jan-2016]. [4] V. Osmani, S. Balasubramaniam, and D. Botvich, Human activity recognition in pervasive health-care: Supporting efficient remote collaboration, J. Netw. Comput. Appl., vol. 31, no. 4, pp , Nov

5 [5] Ericsson Mobility Report: 70 percent of world s population using smartphones by 2020, Ericsson.com, 03-Jun [Online]. Available: [Accessed: 09- Jan-2016]. [6] A. Mannini and A. M. Sabatini, Machine Learning Methods for Classifying Human Physical Activity from On-Body Accelerometers, Sensors, vol. 10, no. 2, pp , Feb [7] D. Anguita, A. Ghio, L. Oneto, X. Parra, and J. L. Reyes-Ortiz, Human Activity Recognition on Smartphones Using a Multiclass Hardware-Friendly Support Vector Machine, in Ambient Assisted Living and Home Care, J. Bravo, R. Hervás, and M. Rodríguez, Eds. Springer Berlin Heidelberg, 2012, pp [8] D. T. G. Huynh, Human Activity Recognition with Wearable Sensors, [9] DetectedActivity, Google Developers. [Online]. Available: /gms/location/detectedactivity. [Accessed: 29-Dec-2015]. [10] ios 9 - Health, Apple. [Online]. Available: [Accessed: 29-Dec-2015]. [11] J. Lester, T. Choudhury, and G. Borriello, A Practical Approach to Recognizing Physical Activities, in Pervasive Computing, K. P. Fishkin, B. Schiele, P. Nixon, and A. Quigley, Eds. Springer Berlin Heidelberg, 2006, pp [12] R. W. de Bettio, A. H. C. Silva, T. Heimfarth, A. P. Freire, and A. G. C. de Sá, Model and implementation of body movement recognition using Support Vector Machines and Finite State Machines with cartesian coordinates input for gesture-based interaction, J. Comput. Sci. Technol., vol. vol. 13, no. 2, Oct