Activity recognition in ADL settings. Ben Kröse

Transcription

1 Activity recognition in ADL settings Ben Kröse

2 Content Why sensor monitoring for health and wellbeing? Activity monitoring from simple sensors Cameras Co-design and privacy issues

3 Necessity for assistive technology

4 Why sensors for health and wellbeing? Assistive technology is needed Physical support Cognitive support Social support Assistive health systems need accurate assessments on the current state of the person; Physical Cognitive Social

5 Why sensors for health and wellbeing? Sensing systems are needed that monitor the patient. Monitoring systems vital signs directly Heart rate blood pressure sugar level, Monitoring the health state indirectly, by measuring the activity behavior of the patient

6 What sort of activities?

7 What sort of sensors?

8 Smart Homes

9 ACTIVITY RECOGNITION FROM SIMPLE SENSORS

10 Monitoring of activities of elderly using simple sensors Psychogeriatric ward Naarderheem Assisted living appartments

11 De Flank: Assisted living homes ambient assisted living

12 What do we measure? Intern: Psychogeriatric ward Naarderheem: 8 rooms with sensors on bed, door, movement detectors. External Assisted living: 7 apartments with 15 sensors each: Bed Diverse kitchen cabinets Doors Electrical appliances Couch Motion sensors

13

14 SENIOR CREATE-IT CHI 2011 VANCOUVER Toilet MAY flush sensor Marije Kanis

15 Long term data collection

16 Feature selection Based on expert knowledge Interview caregivers Make features Select best features Data driven ICA, PCA Clustering

17 Visualization sensordata # Badkamerpatroon:

18 Comparison with measurements by caregivers Occupational therapists use different measures to indicate ability of living independently: Subjective assessment modified KATZ ADL index (self report on basic ADL, instrumental ADL) objective assessment AMPS scale (Physical performance: gait- speedtest 3-m measured walk, grip strenght-test Jamar Dynamometer)

19

20

21 Automatic recognition of activities Try to recognize ADL s from simple sensors

22 Hidden Markov Model Sensor pattern x t depends on activity z t activity z t depends on activity z t-1

23 Hidden Markov Model Activity at time t: z t = {cooking, sleeping,.} Sensor pattern x t : binary vector (0,0,1,1, ) indicating the sensor values at time = t

24 Hidden Markov Model Parameters of the HMM: Transition probability: A=p z t+1 z t Modeled with a matrix A Observation model: p x t z t Assume independence: p x t z t = n p x n,t z t Each feature x n is modeled by an independent Bernoulli distribution, where μ i,n is the parameter of the n th feature p x n,t z t = i = μ i,n x n (1 μ i,n x n ) (1 x n)

25 How to train the parameters? Need a training set consisting of examples: {z 1, x 1, z 2, x 2,..., z N, x N } Estimate parameters with ML methods

26 Inference The inference problem for the HMM consists of finding the single best state sequence that maximizes. p z 1:T, x 1:T Although the number of possible paths grows exponentially with the length of the sequence, the best state sequence can be found efficiently using the Viterbi algorithm.

27 Hidden Markov Model Advantages: Fast and efficient Good precision in recognition Needs relatively little training data Disadvantages: Limited model of the duration of the activities No modeling of the structure or the duration of activities

28 Question What is the probability that the hidden state has a duration d given that the observation is constant?

29 Question What is the probability that the hidden state has a duration d given that the observation is constant p d = a ii d 1 1 a ii

30

31 Conditional random field Don t model the joint probability density but the conditional p z 1:T, x 1:T p z 1:T x 1:T = 1 Z x 1:T φ C z C, x C c C as a product of clique potentials

32 Linear chain conditional random field Clique potentials: p z 1:T x 1:T = 1 T φ t z t, z t 1, x t Z x 1:T t=1 With K φ t z t, z t 1, x t = exp π k f k z t, z t 1, x t k=1

33 Linear chain conditional random field Sensor pattern x t and activity z t are dependent activity z t and activity z t-1 are dependent

34 Conditional Random Field Not a full probabilistic model (more like a neural network) Also training is needed

35 Conditional Random Field Not a full probabilistic model (more like a neural network) Also training is needed

36

37 Conditional random field Advantages: Fast and efficient Good precision in recognition Better model of the duration of the activities Disadvantages: No modeling of the structure or the duration of activities Needs many training samples Slow in training

38 Hidden Semi-Markov Model Dt-1 Dt Dt+1 d t is the remaining duration of a state p(z 1:T, x 1:T,

39 Hidden Semi-Markov Model Advantages: Explicit modeling of the duration of an activity Good precision in recognition Disadvantages: Needs many training samples Computational very expensive

40 Experiments: data

41 Activiteiten

42 Representation

43 Results

44

45

46 Training time

47

48 Cameras vs. other sensors in the home Body worn sensors: Not always worn Stigmatizing Simple ambient sensors Nonintrusive No detailed information Cameras Privacy issues Much information

49 What sort of activities?

50 SIMPLE ACTIONS

51 Simple actions: fall detection Most common cause of injury with persons 55+ In the Netherlands annually emergencies Of which in and around the home 1.3% fatal Problem will increase with ageing population

52 Fall detection: existing solutions Wearable accelerometers Ambient detectors Problems: not worn, restricted use

53 Fall detection with cameras Inactivity measurement Dynamics of the visual features

54 Cameras for fall detection Nait-Charif et al (2004): Inactivity based system: Activity summarisation and fall detection in a supportive home environment}}, author={nait-charif, H. and McKenna, S.J.}, booktitle={proceedings of the 17th International Conference on Pattern Recognition, ICPR 2004},

55 Cameras for fall detection 3D modeling of pose Multiple cameras Time of flight Stereo

56 Fall lab

57

58 Oosterhout et al: 3D dynamics

59 Kinect

60 Compare overhead camera and Kinect

61 Results (40 falls, 40 non-falls in total, 5-fold crossvalidation) Skeleton based Bounding ellipse TP TN FP FN 38.76/ / / / / / / /0.38

62

63 Combining 2 camera s

64 Results (40 falls in total) Skeleton based Bounding ellipse Using all features Feature match TP TN FP FN

65 Conclusions Research shows that methods from AI and computer vision are applicable to the health care domain There is a shortage of supervised data sets There is a shortage of realistic data sets (elderly don t fall like students) Privacy issues have to be taken serious and are a serious problem