Fall Detection System based on Kinect Sensor using Novel Detection and Posture Recognition Algorithm Choon Kiat Lee 1, Vwen Yen Lee 2 1 Hwa Chong Institution, Singapore choonkiat.lee@gmail.com 2 Institute for Infocomm Research, A*STAR, Singapore vyalee@i2r.a-star.edu.sg Abstract. Elderly suffers from injuries or disabilities through falls every year. With a high likelihood of falls causing serious injury or death, falling can be extremely dangerous, especially when the victim is home-alone and is unable to seek timely medical assistance. Our fall detection systems aims to solve this problem by automatically detecting falls and notify healthcare services or the victim s caregivers so as to provide help. In this paper, development of a fall detection system based on Kinect sensor is introduced. Current fall detection algorithms were surveyed and we developed a novel posture recognition algorithm to improve the specificity of the system. Data obtained through trial testing with human subjects showed a 26.5% increase in fall detection compared to control algorithms. With our novel detection algorithm, the system conducted in a simulated ward scenario can achieve up to 90% fall detection rate. Keywords: Fall Detection, Kinect, Posture Recognition 1 Introduction The progressive aging of population has become a major social challenge for countries around the world. As more elderly begin living with health problems and are home-alone, they require increasing assistive support in daily activities. For the elderly, involuntary falls are frequent. Annual statistics show that one in every three adults age 65 and older in the USA have recently suffered a fall [1]. Falls cause a loss in quality of life for the fallen elderly and can be more dangerous due to the fact that the victim can easily lose consciousness and thus become unable to seek help if they are home alone, which is detrimental to their long-term health if the accident is serious and undetected [2]. Thus, in order to avoid this scenario, fall detection 1 Lee Ah Mooi Old Age Home, 1 Thomson Lane, 297728, Singapore
systems that are capable of identifying and notifying caregivers when the elderly fall are very much essential in the bid to provide assistive services to the elderly. Current fall detection systems have fall detection rates of 70% to 80% [3], which might be unconvincing to some consumers when making decisions on purchasing such systems. Other technologies such as the uses of floor sensors [4] have shown fall detection rates of about 90%, but may suffer from other problems that limit its use such as vulnerability to spillages, etc. We aim to develop and test a fall detection system that utilizes the Microsoft Kinect sensor and our novel fall detection algorithm to determine automatically if a fall has occurred. We propose our fall detection system which is optimized using current preliminary datasets and when used in conjunction with our novel posture recognition algorithm, help us to achieve a significantly higher specificity rate of fall detection, up to 90% accuracy rate in an actual test with a real human subject. 2 Fall Detection System In this project, we aim to create a fall detection system (Fig. 1 & 2) for use in confined areas like the bedrooms or washrooms in single room apartments. This constraint is in place as the project is targeted towards the detection of falls for homealone elderly who normally live in such apartments. The Kinect sensor is placed in a tilted position to maximise its coverage of the room, although one or two more sensors could also be employed to cover all blindspots. Our system is also able to operate in situations with little or no light, therefore making it possible for the Kinect to track human activity both in the day and night[5], something that conventional stereo video camera systems are unable to do. This makes it especially useful as most falls by the elderly happen at night when visibility is poor, improving the effectiveness of our fall detection system. Depth images of the surroundings captured by the Kinect are processed by the Microsoft Kinect SDK to return a pipeline of information that provides our fall detection system with skeletal data. Up to 2 user's skeletons can be tracked, with the skeletal data of the user then subsequently passed to our algorithm as described in section 3 for further processing. If a fall is detected, we would relay a signal to an alarm system to sound an alert. Fig. 1: Setup of our fall detection system Fig. 2: Block diagram of our fall detection system
3 Fall Detection Algorithm 3.1 Our Fall Detection Algorithm Skeletal data received from the Kinect sensor is processed by functions 1 & 2. If both functions return a positive result, a fall is tentatively indicated. After that, our postural recognition algorithm is then applied to reduce the amount of false positives returned by our fall detection algorithm, resulting in a fall detection algorithm that manages to achieve relatively high sensitivity and specificity. A preliminary dataset of 10,479 frames of skeletal data from the Kinect comprising 34 fall and non-fall events were used to optimize our enhanced fall detection algorithm 3.2.1 Function 1 (Checking position of user s centre of mass) Our algorithm checks if the real world coordinates of the user s hip centre (roughly representative of the person s centre of mass) obtained from the Kinect is within a certain threshold distance from the floor. 4.2.2 Function 2 (Checking velocity of user s centre of mass) Many current algorithms that make use of the centre of mass are unable to accurately detect slow falls because the measured vertical velocity in slow falls tends to be relatively lower than that measured during a normal fall. Therefore pre-set threshold values are normally not exceeded and the slow fall is not detected. Figure 3 shows the different velocities of the user s hip centre joint where the vertical velocity of the slow fall did not exceed the threshold, resulting in no falls being detected. Our algorithm separates the velocity of the user s hip centre into two components. The first component is the vertical velocity of the hip centre, while the second component is the velocity of the hip centre with respect to the vector sum of the displacement of the hip centre in both the x and z directions, thus giving us the horizontal component of the hip centre s velocity. Fig. 3: Graph showing different velocities of the user s hip centre joint for both the slow fall (solid lines) and the vertical fall (dotted line). Fig. 4: Diagram fall detection rates when the threshold corresponding to score 3 in table 1 is varied. As the value of VV<1.9 gave us one of the highest fall detection rates it was used in our table of scores (table 1)
The different velocities of the user s hip centre are given different scores for the vertical and horizontal velocities (an example is shown in Table 1). Our fall detection algorithm checks if the combined scores for both the vertical and horizontal component exceed our pre-set overall threshold score. The table of scores shown in Table 1 was optimized by varying the parameters individually (Fig. 4) until a maximum fall detection rate on the preliminary dataset of falls was obtained. Table 1 Table showing how the scoring system was applied for vertical velocities Score Vertical velocity of user s hip centre, VV / m s -1 Remarks 0 VV > -0.9 Representative of negligible vertical velocity typical of everyday activities Representative of a small vertical velocity that is typical of a slow horizontal fall, 1-0.9> VV > -1.4 but not low enough to be triggered by everyday activities 2-1.4> VV > -1.6 Representative of a moderate vertical velocity typical of a fast horizontal fall 3 VV < -1.6 Representative of a large vertical velocity typical of a vertical fall This new method of fall detection provides two main benefits: 1. It is more sensitive to falls in any direction than conventional fall detection systems as both the horizontal and vertical components of velocity of the user s hip centre are taken into account 2. It is less prone to false positives generated by large horizontal movements by the user than conventional fall detection systems. 4.2.3 Postural Recognition Algorithm Due to the multiple similarities between falls that are classified as false positive and actual real falls, most current fall detection systems have difficulty differentiating between these two classifications [6]. We demonstrate this similarity with results in Figure 5 using our own data, where the monitored subject s velocity is similar for an actual fall and an action of just bending down. To resolve this problem, we developed a novel postural recognition algorithm that separates the false positives from actual real falls by checking the user s posture to determine the likelihood that the user has been involved in a fall when previous functions 1 and 2 return positive results. Classified Posture using rule no. Sitting on the floor both legs 1 1 folded behind 2 Kneeling on the floor 1 3 Squatting 1 Bending down (to wear shoes 2 4 or tie shoelaces) Fig. 5: Graph showing different velocities of the user s hip centre joint for bending down (solid lines) and for an actual fall (dotted line). Table 2: List of false positive postures considered *Feature 1: User s Ankles have to be below the user s hip centre Feature 2: One of the user s legs is either folded below his body or one knee is significantly higher than the other
In order for our postural recognition system to correctly identify these postures, two main defining features were identified in order to separate fallen postures from other possible postures. These two features were refined and checked against previously obtained skeletal data of postures to ensure their specificity. 4 Results In order to test the actual effectiveness of our system, we conducted a trial experiment that was simulated as closely to the actual conditions where our fall detection system would be deployed. We want to test the robustness of our fall detection system by conducting tests within a setting that is previously unknown to the system. A nursing ward was recreated within our laboratory to provide a realistic setting that is similar to wards found in Lee Ah Mooi Old Folk s Home. A hospital bed was placed in the corner of our ward, with PVC pipes representing the walls of the ward and the Kinect sensor was deployed at the top corner of our ward. The three kinds of falls that frequently occur within the Home were then simulated by our volunteers to replicate the kind of conditions that our fall detection system will face when deployed at the Home. Fig. 6.1 shows our experimental setup, while Fig. 6.2 shows a screenshot of our fall detection system user interface during the experiment. Fig. 6.1 Picture showing experimental setup Fig. 6.2 Screenshot of fall detection system user interface running on laptop Our fall detection system is capable of processing incoming sensor information and displaying of results concurrently without a loss in processing speed or performance degradation. By connecting an SMS gateway or alert module to the system, we can detect falls within half a second of the fall and subsequently alert the caregivers. The system automatically starts when someone enters the field of vision of the Kinect sensor. By producing a stickman display (Fig. 6.2) for display purposes which users can choose to hide, we also managed to preserve the user s privacy. Initial results show that most falls that include falling from the bed or in open spaces were accurately identified by our fall detection system. Multiple other kinds of non-fall events were also accurately detected by our fall detection algorithm and correctly classified, mainly due to the assistance from our novel posture recognition algorithm that points out abnormalities in the data obtained through the sensors, thus demonstrating the usefulness of our posture recognition algorithm in recognizing and
differentiating falls. Our system is able to achieve an overall specificity rate of up to 90% and our algorithms helped improve the fall detection rate such that we could detect all falls (100%) in certain situations, where conditions are optimal. This includes situations where more sensor information allows us to perform inference in order to provide a more accurate assessment. However, further analysis also showed that there were occasions where the system was not able to classify actual falls. For example, a user might fall off the chair and the fall is undetected, due to the fall taking place in a low position, and reduced time required by the system to measure a significant increase in user s velocity while he is falling, hence resulting in the system not being able to correctly detect the fall. We have detailed data (Fig. 7) and information regarding this kind of misclassification and also have a solution for this problem and are currently working on correcting this issue so as to ensure that it does not occur again in subsequent modifications. 5 Conclusions Our fall detection system is able to detect and identify falls using the Kinect sensor within confined spaces at a relatively high specificity rate when compared to usage of control algorithms. The system also uses our novel posture recognition algorithms to enhance our detection rate and ensures robustness during deployment. Overall, with sufficient sensor information while maintaining user s privacy, our fall detection system is able to automatically recognise falls and notify healthcare services to provide timely assistance to elderly who have fallen. References [1] George, F. F. (2000). Falls in the elderly. Am Fam Physician, 61(7), (pp. 2159-2168.) [2] Wild, D., Nayak, U. S., & Isaacs, B. (1981). How dangerous are falls in old people at home?. British medical journal (Clinical research ed.), 282(6260), 266 [3] Perry, J. T., Kellog, S., Vaidya, S. M., Youn, J. H., Ali, H., & Sharif, H. (2009, December). Survey and evaluation of real-time fall detection approaches. In High-Capacity Optical Networks and Enabling Technologies (HONET), 2009 6th International Symposium on (pp. 158-164). IEEE. [4] Rimminen, H., Lindstrom, J., Linnavuo, M., & Sepponen, R. (2010). Detection of falls among the elderly by a floor sensor using the electric near field. Information Technology in Biomedicine, IEEE Transactions on, 14(6), 1475-1476. [5] Yu, X. (2008, July). Approaches and principles of fall detection for elderly and patient. In e-health Networking, Applications and Services, 2008. HealthCom 2008. 10th International Conference on (pp. 42-47). IEEE. [6] Gadre, K. R. (2012). Fall Detection System Using Low Cost Computing (Doctoral dissertation, University of Missouri).