A 3D Motion Trajectory Signature Identification for Human- Computer Interaction (HCI) Applications

Transcription

1 Internationa Journa of esearch Studies in Science, Engineering and Technoogy Voume 2, Issue 7, Juy 2015, PP ISSN (Print) & ISSN X (Onine) A 3D Motion Trajectory Signature Identification for Human- Computer Interaction (HCI) Appications S. M. Youssef Computer Engineering Department, University Arab Academy for Science and Technoogy, Aexandria, Egypt sherin@aast.edu.com N. F. Attia Computer Engineering Department, Pharos University, Aexandria, Egypt neha.fathi@pua.edu.com S.M. Ekaffas Computer Science Department, University Arab Academy for Science and Technoogy, Aexandria, Egypt saeh.mesbah@gmai.com Abstract: Trajectory tracking is used to keep tracks objects. Trajectory tracking contro is used to affect desired trajectories of a device, human and anything can move. In order to precisey track specified trajectories, or be abe to foow more genera trajectories, many tracking contro agorithms have been proposed, but sti there is some probems of tracking trajectory in its appication. This paper introduces a new mode for trajectory signature tracking that integrates the maximay stabiity extrema regions (MSE) feature extraction of first eve transformed domain descriptors with K- nearest neighbour cassifier. Experiments have been carried out on arge datasets of tracking trajectories with different characteristics. MATLAB software version 2013a is used to impement and test the proposed method. Experimenta esuts showed that the proposed mode produces satisfactory performance. Significant improvements have been iustrated in terms of recognition rate and recognition efficiency. Keywords: Trajectory tracking, Motion trajectory, maximay stabiity extrema regions (MSE), K- Nearest neighbour cassifier. 1. INTODUCTION Trajectory tracking contro means tracking reference trajectories predefined or given by path panners. It has been widey [17] studied and various effective methods and tracking controers have been deveoped. Trajectory tracking, measuring the positions and engths of the stationary points is aso usefu for perceiving the characteristics of the pause. Trajectory tracking is reated to ine simpification, on the one hand, and protocos for tracking an object s current position, on the other hand. For carity, we refer to the word as (rea-time) position tracking [11]. This paper addresses the motion trajectory matching probem. Trajectory based motion recognition is one of the most important objectives in motion characterization. The recognition mode, accuracy and efficiency are the three key factors reated to the configurations of trajectory representation and recognition engine. ecognition modes can be categorized according to the initiaization modes. Motion trajectory is a compact and robust cue for motion characterization and it has been extensivey studied for describing actions, behaviors and activities in different appications. No matter for simpe actions, median behaviors or compex activities, the consequent motions can be characterized by identifying the invoved subjects (human body, head, hands, or feet) and extracting the underying motion trajectories [1]. That is, motion can be anayzed spatio-temporay by the joint description of spatiay parae and/or temporay sequentia motion trajectory compositions. Motion trajectories are invoved in many different situations where various requirements on motion characterization exist [20]. The basis to account for the different requirements ies in a fexibe trajectory description. aw trajectory data can be used directy. However, such a simpe way of using the raw data is quite infexibe because it reies much on the absoute positions of the data points. Instead, buiding trajectory representations using some intrinsic properties woud be a better way to capture the shape features of a trajectory and to avoid suffering from the constraints of absoute Internationa Journa of esearch Studies in Science, Engineering and Technoogy [IJSSET] 102

2 A 3D Motion Trajectory Signature Identification for Human-Computer Interaction (HCI) Appications coordinates. Furthermore, deveoping dynamic and adjustabe trajectory representation structures (for exampe, changeabe in size) can benefit the adaptabiity of trajectory descriptions. Trajectory tracking has many appications in our ife ike, hand tracking, eye tracking, trajectory tracking of wheeed mobie appication, motion tracking, etc. Motion trajectories can be extracted by visua tracking from the motions of humans, robots or other moving objects. It might be a singe trajectory tracked from an individua tracking point of interest, or a set of mutipe, interreated, concurrent trajectories extracted from a compicated motion. In fact, a arge body of work about motion tracking can be found in the iterature [1]. This research work just reasonaby assumes that the motion trajectories have been extracted from motions. This paper is organized as foows. Section two represents the background and reated concepts; section three represents the reated word; section four iustrates the proposed mode; section five demonstrates the Excrementa resuts; finay section six gives concusion. 2. BACKGOUND AND ELATED CONCEPTS This section provides the main reated concepts and key terms Signature-Based Trajectory Description For a free form 3-D motion trajectory Γ (t) is parameterized by: [20] Γ (t) = {X (t), Y (t), Z (t) t [1, N]} Where, t is tempora index and N is trajectory ength. Its 3-D Eucidean signature S is defined in terms of differentia invariants: curvature (κ), torsion (τ) and the first order derivatives with respect to Eucidean arc-ength s (κs = dκ/ds and τs = dτ/ds), in the foowing form, [20] S= {[κ (t), τ (t), κ s (t), τ s (t)] t [1,N]} (1) In practice, an approximate signature S* was numericay impemented in terms of the joint Eucidean invariants (the inter-point Eucidean distances) [20]. This not ony enhances the signature s robustness by avoiding cacuating the noise sensitive high order derivatives in the accurate signature, but aso eiminates the requirement to the differentiabiity of the signature components. As the joint invariants are oca features, the signature admits the computationa ocaity that is the basis to deduce substantia descriptive invariants, and that makes the signature insensitive to occusion. From the above definition, it is observed that the signature is based on the features extracted from a the samped trajectory points. Hence it is a compete description to the entire raw trajectory data, which is abe to capture and preserve most motion features. Thus the fitting inaccuracy is not a probem for the signature descriptor [12]. To ensure the existence of the signature mathematicay, it is assumed that the trajectory denoted by Γ (t) is reguar, say, for a t, Γ (t) = 0. For the representation of irreguar trajectories, the stationary points wi be firsty detected by examining the condition of Γ ( t) = 0 and then the trajectory is reguated to be reguar by removing the stationary points. Due to continuous tracking, this means eaving one and removing the extra points among a set of consecutivey samped points that admit stationary points. Accordingy, it is abe to generate a singe signature for the reguated trajectory. Meanwhie, the positions and ength of the removed stationary points are saved for the ater use for trajectory reconstruction [20] Motion Trajectory Signature Signature (trajectory) has two eves. Whie the first eve is the fu description to the entire raw trajectory data, the second eve is the condensed description of the first eve signature data [8]. First Leve: Eucidean differentia invariants features are empoyed to buid a oose trajectory signature. As differentia invariants are typica oca features, they are not ony capabe of characterizing trajectory shapes, but aso can offer richer invariants in trajectory representation with respect to rigid, metric and viewpoint change and insensitivity to occusion. Focusing on the detaied motion features captured by the first-eve signature, motion perception to different trajectory instances can be achieved readiy via the visuaization of the noninear inter-trajectory warping paths. The first-eve signature is particuary appropriate for sma scae appication concerning both trajectory recognition and motion perception [8]. Internationa Journa of esearch Studies in Science, Engineering and Technoogy [IJSSET] 103

3 N. F. Attia et a. Second Leve: Second eve is motivated by reducing the redundancy data in the first-eve signature to obtain a compact signature description towards faster trajectory recognition. Appying PCA transform to the first-eve signature, we get the second-eve signature with ess data but preserving most variance of the trajectory features. Further, based on the Gaussian Mixture modeing to the second-eve signature, a Bayesian cassifier is deveoped to repace the time-consuming matching agorithm. These resorts can speed much up the recognition especiay for arger scae database [8]. 3. ELATED WOK Human-Computer Interaction (HCI) is becoming more significant in the ife. With the progress of computer science, the existing physica devices currenty used in the interaction (e.g. keyboard, mouse,) are not satisfying human needs nowadays. Many designers attempt to improve the usabiity and make the interaction more natura and easier. They introduced the technique of Human-to-Human Interaction (HHI) into the domain of HCI to reach this purpose [10]. In this context, using hand gestures is among the richest HHI domain since everyone uses mainy hand gestures to expain ideas when he communicates with others [6]. Communication with hand gestures becomes important and much cearer with the consideration of sign anguage. Sign anguage (SL) is the natura way of communication among deaf persons. As etters and words in natura anguages, we find the corresponding eements on sign anguages, which are the movements, gestures, postures and facia expressions. Many researchers expoited hand gestures in other appication fieds such as hand tracking [2] and interactive computer graphics [3]. Simiary, severa studies have focused on automatic sign anguage recognition so as to faciitate communication between hearing and deaf persons and to improve HCI usabiity [1] [2]. Severa works in the domain of automatic sign anguage recognition have been interested in the manua information such as the trajectories of both hands. In order to recognize human gestures by motion information, many methods based on trajectories and motions have been proposed to anayze these gestures [4]. Bobick and Wison adopted a state-based method for gestures recognizing [5]. They used a number of sampes, for each gesture, to cacuate its main curve. Each gesture sampe point was mapped to the ength of arc at curve. Afterword, they approximated the discretized curve using a uniform ength in segments. They grouped the ine segments into custers, and to match the previousy earning state of sequences and the current input state, they used the agorithm of Dynamic Time Warping. For continuous ASL (American Sign Language recognition), Haynes and Jain (2010) used a viewbased approach [9]. A anguage recognition appication has been deveoped based on 3D continuous sign [14]. They used the Longest Common Subsequence method for sign anguage recognition instead of HMM (Hidden Markov Mode), which is a costy doube stochastic process. A research on automatic recognition of sign anguage mainy focused on the domain of the dependent signer. A itte work has been devoted to independent signer [10][13]. The sign anguage recognition systems based on independent signer have a promising trend in practica systems that can recognize a different signers SL. But the probem of the recognition of signer independent is hard to resove regarding the great obstace from different sign variations. 4. POPOSED MODEL Figure1 shows the proposed mode. It has different phases incuding pre-processing, buid component tree, arrange extrema regions, refining the seection and the ast phase is recognition. This paper is uses the MSE for feature extraction with the first eve of the signature with the advantage of using K-NN search with motion trajectory in recognition. Fig1. Functiona bock diagram of the proposed mode Internationa Journa of esearch Studies in Science, Engineering and Technoogy [IJSSET] 104

4 A 3D Motion Trajectory Signature Identification for Human-Computer Interaction (HCI) Appications 4.1. aw Motion Trajectory A arger trajectory dataset of UCI KDD high quaity ASL is used to test the signature s retrieva performance especiay the improvement in efficiency benefitted from the second-eve signature [18]. The ASL trajectory dataset consists of 95 sign casses, and 27 sampes were captured for each sign. Figure 2 iustrates some trajectories from the dataset Pre-Processing Fig2. Sign sampes of some words The raw motion trajectory is used to convert raw matrix to matrix contains vaues in range from (0 to 1), then dea with this numbers as from 0.0 (back) to 1.0 (fu intensity or white) Buid Component Tree It is the first phase in MSE. MSE a method for a bob detection in images[14][15], bob detection methods are aimed at detecting regions in a digita image that differ in properties, such as brightness or coour, compared to surrounding regions. Informay, a bob is a region of an image in which some properties are constant or approximatey constant; a the points in a bob can be considered in some sense to be simiar to each other. A method for a bob detection in images [14] [15]. Bob detection methods are aimed at detecting regions in a digita image that differ in properties, such as brightness or coour, compared to surrounding regions. Informay, a bob is a region of an image in which some properties are constant or approximatey constant; a the points in a bob can be considered in some sense to be simiar to each other. This method of extracting a comprehensive number of corresponding image eements contributes to the wide-baseine matching, and it has ed to better stereo matching and object recognition agorithms MSE is a feature detector; Like the SIFT detector, the MSE agorithm extracts from an image I a number of co-variant regions, caed MSEs. An MSE is a stabe connected component of some eve sets of the image I. optionay; eiptica frames are attached to the MSEs by fitting eipses to the regions; for a more in-depth expanation of the MSE detector. A stabe region has a sma variation. The agorithm finds regions which are "maximay stabe", meaning that they have a ower variation than the regions one eve beow or above. Note that due to the discrete nature of the image, the region beow or above may be coincident with the actua region, in which case the region is sti deemed maxima. In genera, an extrema region is a set of pixes connected by their 4-neighbors that satisfy the property that a of their intensities are uniformy greater or ess than the intensities of every pixe that surrounds the region. An extrema region is maximay stabe if given intensity i and a margin, the change in the number of pixes in the region from i to i + is a oca minimum. A arge vaue for wi generate ony a few highy stabe regions, whie a smaer vaue wi detect many ess-stabe regions. The advantage of a sma deta vaue is that more regions wi be detected which aows more invariant information to be extracted from an image [16]. Component tree is a representation of a gray-eve image that contains information about each image component and the inks that exist between components at sequentia gray-eves in the image, a pixes are arranged by their intensity and neighborhood reationship. Every chid tree is corresponding to a region, and the root of the chid tree is the index of the pixe that has the biggest vaue in the region. Internationa Journa of esearch Studies in Science, Engineering and Technoogy [IJSSET] 105

5 N. F. Attia et a Arrange the Extrema egions It the second stage in MSE. Extrema regions are the nodes whose parent nodes have a bigger vaue. Arrange the extrema regions in a tree of nested regions, as shown in Figure 3. Connecting two regions _ and _+1, if and ony if B,...,, (2),..., min 1 max Fig3. Extrema regions are arranged in a tree of nested regions [19],, the stabiity score is: [19] 1 B,..., v Seect the maximay stabe region in the branch minima stabiity score is (3), 1, which has a oca B,..., If v < v 1 ( If v > v 1 ( Otherwise, do nothing 1 ), mark ), mark 4.5. efining the Seection as unstabe as unstabe It is the ast stage in Maximay Stabe Extrema egion (MSE).In this stage we finaize the Maximay Stabe Extrema egion agorithm by: emove very sma and very big regions emove regions which have too high area variation emove dupicated regions 4.6. ecognition In the this phase K-NN cassifier is used to find the distance between the regions orientation, the most cose signs gives us ower distance than others, the next tabes shows some orientation distances to show the accuracy of the proposed mode. The k-nearest Neighbours agorithm is a non-parametric method used for cassification and regression. In both cases, the input consists of the k cosest training exampes in the feature space. The output depends on whether k-nn is used for cassification or regression. In k-nn cassification, the output is a cass membership. An object is cassified by a majority vote of its neighbors, with the object being assigned to the cass most common among its k nearest neighbors (k is a positive integer, typicay sma). If k = 1, then the object is simpy assigned to the cass of that singe nearest neighbour. In k-nn regression, the output is the property vaue for the object. This vaue is the average of the vaues of its k nearest neighbours [7]. KNN for cassification, in this case, we are given some data points for training and aso a new unabeed data for testing. Our aim is to find the cass abe for the new point. The agorithm has different behavior based on k, for exampe: k = 1, this is the simpest scenario. Let x be the point to be abeed. Find the point cosest to x. Let it be y. Now nearest neighbor rue asks to assign the abe to x. This seems too simpistic and sometimes even counter intuitive. 5. EXPEIMENTAL WOK A arger trajectory dataset of UCI KDD high quaity ASL [18] is used to test the trajectory recognition performance in terms of two important measurements: accuracy and efficiency. The ASL trajectory Internationa Journa of esearch Studies in Science, Engineering and Technoogy [IJSSET] 106

6 A 3D Motion Trajectory Signature Identification for Human-Computer Interaction (HCI) Appications dataset consists of 95 sign casses (words), and 27 sampes were captured for each sign. We used haf the sampes for training and the other haf for testing. Severa casses of sampes were randomy seected and recognized in our proposed method. Moreover, different resuts from the scheme have been compared with resuts of other agorithms in [8] [20]. Probem Find the most stabe region from motion trajectory. Soution Motion trajectory are set of positions ((x1,y1,z1),(x2,y2,z3),..(xn,yn,zn)). Convert the matrix of ocation to gray scae matrix, smaest refect back pixes, argest refect white pixes. Then represent the gray scae matrix as a gray scae image as shown in Figure 4 and compute a previous steps showed in 4.3, 4.4 and 4.5 to extract the MSE region from the motion trajectory. (a) Sign sampe of a aive word (b) Aive word as an intensity image Fig4. Sign sampe with its corresponding intensities eves 5.1. Test and Vaidation We appy some cases as a sma exampe beong to sma database to meet our used experiment, in this exampe the database contains a sign sampes from two casses (aive, answer) for exampe, each cass contain 3 signs Case One The test sign is from one of these sampes from the database to check vaidation using the word aive, sign number three. Tabe1. Distances between Train Words and Test Word for Case One Cass one (Answer ) Cass two (Aive) answer-1 & aive-3 & aive-3 answer-3 & aive-3 Aive-1 & aive-3 Aive-2 & aive-3 Aive-3 & aive Fig5. The distance between train sampe casses and a test cass sampe that describe Tabe 1 Internationa Journa of esearch Studies in Science, Engineering and Technoogy [IJSSET] 107

7 N. F. Attia et a Case Two The test sign is from one of these sampes from the database to aso check vaidation using the word answer sign number two. Tabe2. Distances between Train Words and Test Word for Case Two Cass one (Answer ) Cass two (Aive) answer-1 & & answer-3 & Aive-1 & Aive-2& Aive-3& Fig6. The distance between train sampe casses and a test cass sampe that describe Tabe Case Three The test sign is a sign beongs to the casses but not in the database to check verification, the test sign is beongs to cass two. Tabe3. Distances between Train Words and Test Word for Case Three Cass one (Answer ) Cass two (Aive) answer-1 & aive & aive answer-3 & aive Aive-1 & aive Aive-2& aive Aive-3& aive Fig7. The distance between train sampe casses and a test cass sampe that describe Tabe 3, the recognized cass is cass number two, and speciay is recognized to aive-1cass Case Four The test sign is a sign not beongs to any cass in the database to check verification, the test sign is beongs to cass (word) eat. Tabe4. Distances between Train Words and Test Word for Case Four Cass one (Answer ) Cass two (Aive) answer-1 & eat & eat answer-3 & eat Aive-1 & eat Aive-2& eat Aive-3& eat Internationa Journa of esearch Studies in Science, Engineering and Technoogy [IJSSET] 108

8 A 3D Motion Trajectory Signature Identification for Human-Computer Interaction (HCI) Appications The output of the previous case is that cass is not recognized to any cass becouse it doesnot match our verification test.the proposed mode achive 100 % for vaidation and it has a high accuracy with respect to agorithm in [8][20] that shown in Tabe 5, and it has a higher efficiency (in terms of query time), as the casses number increases, the efficiency decreases much sower as shown in Tabe 6.The proposed mode behaves generay better on both accuracy and efficiency with respect to previous agorithms in [8][20]. Tabe5. Average atio of Correct ecognition Cass No Proposed mode 97% 95% 92% The second-eve signature[8] 92.38% 86.17% 78.66% Fu signature based GMM and Bayesian recognition[21] 92.73% 88.58% 83.02% Fu signature and DTW matching [21] 90.03% 87.52% 91.19% Tabe6. ecognition Efficiency Comparison (Units MS/Query) Cass No Proposed mode The second eve signature [ 8 ] Fu signature based GMM and Bayesian recognition[20] Fu signature and DTW matching [20] CONCLUSION This paper presented a new method for 3D motion trajectory that integrates the maximay stabiity extrema regions (MSE) feature extraction of first eve transformed for main descriptors with K- nearest neighbour cassifier. The proposed mode achieves efficient and high recognition accuracy compared with previous agorithms. EFEENCES [1] Voger, C.P.: American Sign Language recognition reducing the compexity of the task with phoneme-based modeing and parae hidden markov modes. PhD thesis, University of Pennsyvania, Phiadephia, PA, USA. Supervisor-Dimitris N. Metaxas, [2] Lefebvre-Abaret, F., Automatic processing of videos FSL: Modeing and operating phonoogica constraints of movement PhD thesis, University of Tououse III - Pau Sabatier, [3] Freeman, W.T., Anderson, D., Beardsey, P., Dodge, C., Kage, H., Kyuma, K., Miyake, Y. Computer vision for interactive computer graphics, in Proc. IEEE Computer Graphics and Appications, May-June, 1998, pp [4] Johansson, G. A Visua Perception of Bioogica Motion and a Mode for Its Anaysis, Perception and Psychophysics, vo. 73, no. 2, 1973, pp [5] Bobick, A.F., Wison, A.D. A State-Based Approach to the epresentation and ecognition of Gesture, in ptoc. IEEE Trans. Pattern Anaysis and Machine Inteigence, Dec. 1997, vo. 19, no. 12, pp [6] Voger, C., Metaxas, D. A ASL ecognition Based on a Couping between HMMs and 3D Motion Anaysis, in Proc. Sixth IEEE Int Conf. Computer Vision, 1998,pp [7] Sitong Liu, Jianhua Feng, Yongwei Wu, egion-aware Top-k Simiarity Search, Lecture Notes in Computer Science Berin, Germany: Springer Voume 9098, 2015, pp [8] Shandong Wu, Y. F. Li and Jianwei Zhang, A Hierarchica Motion Trajectory Signature Descriptor, in Proc. IEEE Internationa Conference on obotics and Automation, 2008, pp [9] Haynes, S., Jain,., A Detection of Moving Edges, Computer Vision, Graphics, and Image Understanding, vo. 21, no. 3, pp , 1980 [10] Maher Jebai, Patrice Dae, Mohamed Jemni, Extension of hidden markov mode for recognizing arge vocabuary of sign anguage, Internationa Journa of Artificia Inteigence & Appications, Vo. 4 Issue 2, p35, Mar2013. Internationa Journa of esearch Studies in Science, Engineering and Technoogy [IJSSET] 109

9 N. F. Attia et a. [11] aph Lange, Frank D urr, Kurt otherme, Efficient ea-time Trajectory Tracking, VLDB Journa, Vo.20,pp , [12] S. D. Wu, Y. F. Li and J. W. Zhang, Signature descriptor for free form trajectory modeing, in Proc. IEEE Internationa Conference on Integration Technoogy, Shenzhen, China, 2007, pp [13] Liang,.H., Ming, O. A Sign Language ecognition System Using Hidden Markov Mode and Context Sensitive Search, in Proc. of ACM Symposium on Virtua eaity Software and Technoogy, 1996, pp [14] Jabaah, K., Jemni,M., Toward Automatic Sign Language ecognition fromweb3d Based Scenes, ICCHP, Lecture Notes in Computer Science Berin, Germany: Springer,vo.6190, Juy [15] Zaman Khan,., Adnan Ibraheem, N., Hand Gesture ecognition: A Literature eview, Internationa Journa of Artificia Inteigence & Appications (IJAIA), Vo.3, No.4,pp , Juy [16] J. Matas, O. Chum, M. Urban, and T. Pajda. obust wide baseine stereo from maximay stabe extrema regions, In 13th BMVC, September 2002, pages [17] azvan Soea, Urbano Nunes, Trajectory Panning and Siding-Mode Contro Based Trajectory-Tracking for Cybercars, Integrated Computer-aided Engineering, Int. Journa, IOS Press, vo.14, n.1, pp.33-47, [18] UCI KDD ASL Archive, [Onine], Avaiabe: htm. Last accessed on 29/6/2015. [19] Marc O Maey Hussein A Hadad, Automatic egistration of Large Images from Light Microscopy, M.E. thesis, Lund University Facuty of Engineering. [20] Shandong Wu and Y.F. Li, "Fexibe Signature Descriptions for Adaptive Motion Trajectory epresentation, Perception and ecognition," Pattern ecognition, vo. 42, no. 1, pp , Jan AUTHOS BIOGAPHY Neha Fathi is a teaching assistant in the department of Computer Engineering, Pharos University, Aexandria. Egypt. She received her B.Sc. degree in Computer Engineering from Arab Academy for Science and Technoogy in Sherin M. Youssef, Professor, head of computer Engineering Department. She obtained her Ph.D., Mphi from University of Nottingham, UK (2004) in the fied of Inteigent Systems, and M.Sc. from University of Aexandria (1996) in Machine Learning & Pattern Anaysis. Her research interest ies in the fied of signa processing, video/image processing, image coding, inteigent systems, signa compression, Biomedica Engineering, Biometrics, QoS inteigent networks, Machine earning, Genetic agorithms, Neuro-based systems, Swarm Inteigence, and Mutimedia systems. Saeh Mesbah Ekaffas, Associate Professor at the Dept. of IS, Coege of Computing & IT, Arab Academy for Science, Technoogy & Maritime Transport (AASTMT). He is currenty the director of the emote Sensing & Spatia Studies Unit, AASTMT. He received his B.Sc. in Eectronics & Teecommunications Engineering, M.Sc. and Ph.D. from Aexandria Univ. His research interests incude digita image processing, sateite remote sensing, geographic information systems (GIS) and environmenta studies. Internationa Journa of esearch Studies in Science, Engineering and Technoogy [IJSSET] 110