Persian Sign Language Recognition Using Radial Distance and Fourier Transform

Transcription

1 I.J. Image, Graphics and Signal Processing, 204,, Pblished Online November 203 in MECS ( DOI: 0.585/ijigsp Persian Sign Langage Recognition Using Radial Distance and Forier Transform Bahare Jalilian Department of Compter Engineering, Kermanshah Science and Research Branch, Islamic Azad University Kermanshah, Iran Abdolah Chalechale Department of Compter Engineering, Razi University Kermanshah, Iran Abstract This paper provides a novel hand gestre recognition method to recognize 32 static signs of the Persian Sign Langage (PSL) alphabets. Accrate hand segmentation is the first and important step in sign langage recognition systems. Here, we propose a method for hand segmentation that helps to bild a better vision based sign langage recognition system. The proposed method is based on YCbCr color space, single Gassian model and Bayes rle. It detects region of hand in complex backgrond and non-niform illmination. Hand gestre featres are extracted by radial distance and Forier transform. Finally, the Eclidean distance is sed to compte the similarity between the inpt signs and all training featre vectors in the database. The system is tested on 480 postre images of the PSL, 5 images for each 32 signs. Experimental reslts show that or approach is capable to recognize all 32 PSL alphabets with 95.62% recognition rate. Index Terms Persian sign langage, hand gestre recognition, Gassian model, centroid distance, Forier transform, Eclidean distance I. INTRODUCTION Sign langage is a non-verbal visal langage that sed by deaf. In order to improve commnication between deaf and hearing people, research in atomatic sign langage recognition is needed. One application of sign langage recognition systems are the se on Internet becase many deaf people are nable to se the World Wide Web and commnicate by in the way normal people do, since they commonly have great difficlties in reading and writing. The reason for this is that normal people learn and perceive written langage as a visal representation of spoken langage. For deaf people, however, this correspondence does not exist, and letters which encode phonemes are jst symbols withot any meaning. The major part of sign langage is comprised of gestres and postres hand that imply meaningfl information. In gestre recognition, sign Langage is translated for dynamic hand motions. In postre recognition, sign Langage is recognized for static alphabets and nmbers. Sign langage recognition system is sefl for the hearing impaired to commnicate with the normal people. Research on atomatic sign recognition has been started since arond 995 []. Many different techniqes for hand gestres recognition have been analyzed, sch as fzzy logic [2], Hidden Markov Models (HMMs) [3], neral networks [4, 5] and spport vector machine [6]. Tsai and Hang sed Spport Vector Machine (SVM) to recognize the static sign and apply HMM model to identify the dynamic signs in the Taiwanese Sign Langage (TSL) [6]. Mehdi and Niaz khan [7] have proposed an American Sign Langage Recognition based on sensor glove to captre the signs. Artificial neral networks are sed to recognize the sensor vales coming from the sensor glove. Lee and Tasi have sed 3D data and neral network to interpret Taiwan sign langage [8]. Assaleh et al [9] have proposed the first continos (Arabic Sign Langage) ArSL system that was able to recognize ArSL sing HMM. Hogh transform and neral network have been sed for recognition American Sign Langage [0]. In [] a system for Arabic Sign Langage Recognition is designed that based on a Gassian skin color model to detect the signer's face and a Hidden Markov Model to recognition signs. Wavelet transform and neral network have been sed for recognition dynamic signs of American Sign Langage [2]. A system for Arabic Sign Langage Recognition is provided which the inpt image is converted into YCbCr color space and a skin profile is sed to detect the skin color from the YCbCr image. Principal Component Analysis algorithm is sed to compose the featre vectors for signs and gestres library [3]. Kiani Sarkaleh et al [4] proposed a Persian sign langage recognition system which is capable of recognizing 8 Persian signs by discrete wavelet transform and a mlti layered Perceptron (MLP) Neral Network (NN) to classify the selected images. In this system backgrond of all images was black. Karami et al [5] designed a

2 Persian Sign Langage Recognition Using Radial Distance and Forier Transform 4 system to recognize static signs of the Persian Sign Langage (PSL) alphabets, backgrond of all inpt images were niform and black. Hand images are converted to grayscale images. Their system is based on the wavelet transform and neral networks. Palraj et al [6] introdced a very simple Malaysian Sign Langage recognition system based on the area of the objects in a binary image and Discrete Cosine Transform (DCT) for extracting the featres from the video sign langage. In preprocessing stage the movie frames are converted into indexed image format and Average filter is applied on these images and the nwanted noises are removed. A simple sign langage recognition system was developed sing the skin color segmentation, moment invariants for featres extracting and neral network model [7]. In most of approaches, inpt hand images have been assmed that have niform and plain backgrond. The signs can be either static or dynamic. A static sign is a particlar hand shape and pose which represented by a single image. A dynamic sign is a moving gestre that represented by a seqence of hand images. The Persian sign langage consists of approximately 075 gestres of the common alphabets and words that the basic Persian sign alphabet is composed of 37 static and dynamic gestres [9]. Or approach focses on the recognition of 32 static hand images. In this paper, we propose a system to interpret static gestres of alphabets in Persian sign langage (PSL). First of all, an effective hand segmentation method is presented to detecting hand region in a complex backgrond with changing illmination condition. The hand segmentation method begins by modeling hman skin color in YCbCr color space sing a database of skin pixels. Skin color distribtion is modeled as a single Gassian model. Similarly a nonskin or backgrond model is bilt sing a database of non-skin pixels. Ths a Skin Probability Image is obtained in which the gray level of each pixel represents the probability of the corresponding pixel in the inpt image to represent skin. Next, hand gestre featres are extracted by radial distance and Forier transform. The rest of the paper is organized as fo llo ws. A detail description of the proposed PSL recognition system is presented in Section Ⅱ. In Section Ⅲ, we exhibit or experimental reslts with discssion. Finally, the conclsions and frther work are presented in Section Ⅳ. II. THE SUGGEST ED SYST EM The sggested system is provided to recognize 32 static sign of the Persian sign langage (PSL) alphabets. These signs are shown in Figre.We assmed that the inpt images inclde exactly one hand. Or system has two main phases: hand segmentation phase (hand region detection) and the featre extraction phase. The block diagram of the proposed recognition system is shown in Figre 2 and its main steps are discssed in the following. Figre. The Persian Sign Langage Alphabets. A. Hand Segmentation Image Pre-processing and detection of the hand region is necessary for image enhancement and for getting good reslts in sign langage recognition systems. The proposed system introdces a method for

3 42 Persian Sign Langage Recognition Using Radial Distance and Forier Transform segmenting hands sing YCbCr color space, single Gassian model, Bayes rle and morphology operators. The block diagram of this phase is shown in Figre 3. between skin and non-skin pixels to classify skin-pixel and to provide robst parameter against varying illmination conditions. RGB vales can be transformed to YCbCr color space sing (): Y Cb = Cr R G 8.24 B () If only the chrominance component is sed, segmentation of skin colored regions becomes powerfl in this process. Therefore, the variations of lminance component are eliminated as mch as possible by choosing the CbCr plane (chrominance components) of the YCbCr color space to bild the model. Research has shown that skin color is clstered in a small region of the chrominance space [4], as shown in Figre 4. Figre 2. The Block Diagram of the Proposed Recognition System Figre 4. Skin color Distribtion in YCbCr Color Space Figre 3. The Block Diagram of the Segmentation Phase in the Proposed Recognition System ) YCbCr Color Space Conversion One important factor that shold be considered while bilding a statistical model for color is the choice of a right color space. For most images, the RGB color space is considered as the defalt color space. In order to convert into other color spaces, we can apply linear or non-linear transformation on the RGB components. In this algorithm, after that color images are resized to 28 by 28 p ixe ls, the inpt RGB image is converted in to YCbCr images de to the fact that RGB color space is more sensitive to different light conditions so we need to transform the RGB vales in to YCbCr. The color space transformation is assmed to decrease the overlap 2) Single Gassian Model The skin color distribtion in CbCr plane is modeled as a single Gassian model. According to section 2. the reason for sing a single Gassian model is the localization of skin color to a small area in the CbCr chrominance space. This step begins with the modeling of skin and non-skin color sing a database of skin and non-skin pixels, respectively. A database of labeled skin pixels is sed to train the Gassian model. Some of skin images from the database are shown in Figre 5. The mean and the covariance of the database characterize the model. Images containing hman skin pixels as well as non-skin pixels are collected. The skin pixels from these images are careflly cropped ot to form a set of training images.

4 Persian Sign Langage Recognition Using Radial Distance and Forier Transform 43 where µ s and Σ s represent the mean vector and the covariance matrix of the training pixels, respectively. Ths the mean and the covariance have to be estimated from the training data to characterize the skin color distribtion as illstrated by (3) and (4). In these eqations, n is nmber of samples in training set. µ = n s j= c j n (3) = n T s = j c j s c j s n ( µ )( µ ) (4) Figre 5. Some of Skin Images from the Skin Database Let c = [Cb Cr] T denote the chrominance vector of an inpt pixel. Then the probability that the given pixel lies in the skin distribtion is given by (2): exp p c skin = 2 ( / ) T ( c µ ) Σ ( c µ ) 2π s Σs s s (2) Then, a Gassian model similar to skin model is bilt for non-skin p ixels also wh ich is called the non-skin or the backgrond model. A database of backgrond region is sed to train the non-skin model. Some of non-skin images from the database are shown in Figre 6. Then the probability that the given pixel lies in the non-skin distribtion is c/non-skin). The skin model and nonskin model are sed to obtain the Skin Probability Image of an inpt color image. Once the skin color and backgrond are modeled sing the single Gassian, these can be sed to calclate the probability of an inpt pixel representing skin, i.e. skin/c), where c is the inpt color vale. c/skin) is again sed to compte the reqired probability skin/c). To compte this probability, the Bayes rle is sed [9]: c / skin) P( skin) skin / c) = (5) c / skin). P( skin) + c / non skin). P( non skin) To calclate the probability, skin/c), for each inpt pixel, The probabilities skin) and non-skin) can be estimated from skin and non-skin image in the training database [9]. In this stdy, for training set we assmed that all p ixels are belong to the skin or non-skin clsters. Hence, we sed: p ( skin) = non skin) = 0.5 (6) c / skin) skin / c) = (7) c / skin) + c / non skin) Ths, the two conditional probabilities and the above ratio are compted pixel-by-pixel to give the probability of each pixel representing skin given its chrominance vector c. This reslts in a gray level image where the gray vale at a pixel indicates the probability of that pixel representing skin. This is called the Skin Probability Image (SPI) given by (8): SPI ( i, j) = skin / cij ) (8) Figre 6. Some of Non-Skin I mages from the Non-Skin Database Where a is a proper scaling factor and c ij is the chrominance vale of pixel (i,j). Here a is chosen to be 255 so that the highest probability vale reslts in a gray level of 255 in the Skin Probability Image. Then gray

5 44 Persian Sign Langage Recognition Using Radial Distance and Forier Transform image obtained is converted into a binary image by Ots thresholding. 3) Morphology Operations The binary image obtained in the previos section may contain white pixels at non-skin regions (backgrond) where the backgrond color resembles the color of skin or black pixels at hand region. These noises may be cased de to bad lighting conditions or existing pixels similar to skin pixels in those regions. In order to detect the hand clearly, it has been frther implemented morphology operations, to fill p the black pixels on the segmented hand and white pixels on backgrond. There are two operations involve namely dilation and erosion. Firstly, dilation operation is performed. Dilation adds pixels to fill p any missing pixels in hand region. Secondly, erosion operation is performed. Erosion removes any white pixels which do not belong to the hand region. This stage is performed to improve the reslt of hand segmentation. Figre 7 shows the images obtained after applying each of steps in the proposed segmentation algorith m. Figre 7(a) is orig inal image; Figre 7(b) is Skin Probability Image. Figre 7(c) is the binary image after Ots thresholding. Figre 7(d) is the hand detected after morphology operation. Figre 8. Implemented Radial Distance Techniqe on Hand Region. The bondary of a shape consists of a series of edge points. A "radial" is a straight line joining the centroid to an edge point. The "centroid" is located at the position (x c, y c ) sch that x c and y c are calclated, respectively, sing (9) and (0). N x x = c = 0 ( ) N N y y = c = 0 ( ) N (9) (0) Lengths of a shape s radial from its centroid, r(), are compted by (). (x(), y()) are coordinates of edge points and (x c, y c ) is coordinate of centroid. 2 2 r( ) = ( x( ) x c ) + ( y( ) yc ) () The signatres generated by this techniqe are invariant to translation, bt they do depend on slope and scaling (hand' size). Normalization with respect to slope and scaling can be achieved by Forier transform sing (2). n a n = r( ) e n t = j2πn N n =,..., N (2) Figre 7. (a): Original Image, (b): Skin Probability Image, (c): Segmented Hand After Ots thresholding (d): The Hand Detected After Morphology Operations. B. Featre Extraction In this phase, first, we apply Sobel edge detector to the hand segmented image, then featres are extracted fro m edges of the hand region. Next, we se radial distance model to obtain a -D fnctional representation of a bondary shapes (signatres) and to bild featre vectors [20]. Radial distance techniqe is based on the distance from the centroid of the shape to the bondary edge pixels as a fnction of angle, as shown in Figre 8. Where, N is the nmber of selected points on the bondary of the hand shape. The compted coefficients, a n, are divided by the maximm coefficient. We considered only first N/2 coefficients of Forier for the final featre vector, that is: a a2 a n / 2 fv = [,,..., ] (3) a0 a0 a0 Therefore, each sign is expressed by these compted coefficients as its featre vector. C. Classification and Recognition Stage A featre vector is compted for each image sign or gestre in the training set and stored in the training data set. When the system receives a new sign, it segments

6 Persian Sign Langage Recognition Using Radial Distance and Forier Transform 45 hand region, detects edges of hand and extracts a featre vector for it. The system comptes the Eclidean distance between featre vector of the inpt image sign and all the stored featre vectors in the training data set. Finally, the sign with minimm distance is selected as the most similar sign and best match. sing a digital camera fro m d ifferent persons, in a complex backgrond with changing illmination conditions and environments. The system is meant to be independent of the distance from the camera. So me samples from or signs database are shown in Figre 9. We divided the database into 2 sets, 320 images are selected for the training set and 60 images are employed as the test set. Hence, the training set is composed of 320 featre vectors. For testing the system, we measred the Eclidean distance between featre vectors of images in the test set and the featre vectors in the training set. Recognition rate of the sggested system is defined as (4): NmberofCorrectlyClassifiedSign Re cognitionr ate = 00% (4) TotalNmberofSigns As we can observe from the Figre 0, the proposed system is capable to recognize 32 PSL alphabets with 95.62% recognition rate which a good reslt is considering the diversity of data in dataset. Figre 0. Recognition Rates of the System for each PSL Alphabets IV. CONCLUSIONS AND FURTHER WORK Figre 9. Some Samples from Or Signs Database III. EXPRIMENTAL RESULT S For experiments, we have sed 32 static signs of PSL alphabets to test validity and robstness of the proposed system. Here, we bilt a database consists of 480 images (5 images of each sign). The sign images are captred This paper proposed a new techniqe for PSL recognition system. The system can detect the hand region in a complex backgrond, changing illmination and environmental conditions and inclding different skin colors. In the first step, the hand region is detected and segmented sing a single Gassian model in YCbCr color space and Bayes rle. Then, In order to find the most effective featres, we sed the radial distance model and Forier transform for featre extraction. As a reslt, the extracted featres for different signs are discriminated while they are invariant to the scaling and slope of the hand shape. The testing reslts showed 00% recognition rate with almost all the 32 signs except 7 signs. Ths, the reslts demonstrated that the proposed system is capable to recognize 32 PSL alphabets with a recognition rate of 95.62%. The

7 46 Persian Sign Langage Recognition Using Radial Distance and Forier Transform proposed approach needs no constraint sch as gloves, sensors and ill minations. In the ftre, we intend to extend the proposed method to constrct a complete Persian sign langage recognition system inclding both static and dynamic signs in order to help deaf and hearing impaired people to commnicate with others. REFERENCES [] M.B. Waldron and S. Kim, " Isolated ASL sign recognition system for deaf persons," IEEE Transactions on Rehabilitation Engineering, 3, 26 27, 995. [2] C.S. Lee, G.T. Park and J.S. Kim, "Real-t ime Recognition System of Korean Sign Langage based on Elementary Components,"997IEEE Proceedings of the Sixth IEEE International Conference on Fzzy Systems. Vol. 3, pp [3] M. AL-Rosan and K. Assaleh, "Video-based signer-independent Arabic sign langage recognition, sing hidden Markov models," Appl Soft Compt 9(3), 2009, pp [4] E. Stergiopolo and N.Papamarkos, "Hand gestre recognition sing a neral network shape fitting techniqe," Engineering Applications of Artificial Intelligence22, 2009, pp [5] P. Vamplew and A. Adams, "Recognition of sign langage gestres sing neral networks," Astralian Jornal of Intelligent Information Processing Systems, 998, vol. 5, [6] B.L. Tsai and C.L. Hang, "A Vision-Based Taiwanese Sign Langage Recognition System, IEEE International Conference on Pattern Recognition, 200, Vol. 5, pp [7] S.A. Mehdi and Y. Niaz Khan, "Sign Langage Recognition Using Sensor Gloves," Proceedings of the 9th International Conference on Neral Information Processing (ICONIP 02), 2002, Vol. 5, pp [8] Z. H. Lee and C. Y. Tsai, "Taiwan sign langage (TSL) recognition based on 3D data and neral networks", Expert Systems with Applications, 2009, vol.36, [9] K. Assaleh, T. Shanableh, M. Fanaswala and F. Amin, "Continos Arabic sign langage recognitionin ser dependent mode", J Intell Learn Syst Appl 2, 200, pp [0] Q. Mnib, M. Habeeb, B. Takrri and AI-Malik, "American sign langage (ASL) recognition based on Hogh transform and neral networks", Expert Systems with Applications, 2007, vol.32, pp [] M. Mohandes and M. Deriche, "Image Based Arabic Sign Langage Recognition", the 8TH International Symposim on Signal Processing and ITS Applications, 2005, pp [2] S. Kmar, D. K. Kmar and A. Sharma, "Visal hand gestre classification sing wavelet transforms, International Jornal of Wavelets", Mltiresoltion and Information Processing, 2003, vol., pp [3] E.E. Hemayed and A. S. Hassanien, "Edge-based recognizer for Arabic sign langage alphabet (ArS2V -Arabic sign to voice)," IEEE International Compter Engineering Conference, 20, pp [4] A. K. Sarkaleh, F. Poorahangaryan, B. Zanj and A. Karami, "A Neral Network Based System for Persian Sign Langage Recognition", IEEE International Conference on Signal and Image Processing Applications, 2009, pp [5] A. Karami, B. Zanj, A. K. Sarkaleh, "Persian sign langage (PSL) recognition sing wavelet transform and neral networks." Expert Systems with Applications 38, 20, pp [6] M.P Palraj, S. Yaacob H. Desa and C.R. Hema, "Extraction of Head and Hand Gestre Featres for Recognition of Sign Langage", International Conference on Electronic Design, 2008, pp.-6. [7] M.P Palraj, S. Yaacob and M. Azalan, R. Palaniappan, "A Phoneme Based Sign Langage Recognition System Using Skin Color Segmentation", 6th International Colloqim on Signal Processing & Its Applications, 200, No. 4, pp [8] I. Bahadori, Persian sign langage collection for the deaf. Edcation and research office rehabilitation research grop. Iran Welfare Organization, 992. [9] V. Vezhnevets, V. Sazonov and A. Andreeva, "A srvey on pixel-based skin color detection techniqes, In Proceedings of the GraphiCon 03, 2003, pp [20] K.L. Tan, and L. F. Thiang, Retrieving similar shapes effectively and efficiently, Mltimedia Tools and Applications, Klwer Academic Pblishers, Netherlands. 2003, pp. 34. Bahare Jalilian, She is crrently working toward the M.Sc. degree in the Islamic A zad Un iversity, Kermanshah Science and Research Branch. She obtained her B.S. in Compter Engineering from Razi University of Kermanshah, Iran in 20. Her research interests inclde compter vision, digital image processing and pattern recognition. Abdol ah Chalecale, Born in Kermanshah, Iran, received his B.S. and M.Sc. degrees in Electrical Engineering (Hardware) and Compter Engineering (Software) from Sharif University of Technology, Tehran, Iran. He received his Ph.D. degree from Wollongong University, NSW, Astralia in 2005 and crrently is with Razi University, Kermanshah, Iran. His research interests inclde image processing, machine vision and hman-machine interactions.