Unobtrusive fall detection and sensor system interoperability for telemonitoring

Transcription

1 Health for All - Nationally And Internationally by Medical Informatics Unobtrusive fall detection and sensor system interoperability for telemonitoring Martin Staemmler Medical Informatics, University of Applied Sciences, Stralsund contact: martin.staemmler@fh-.de understanding reality facing challenges creating future 1

2 Unversity of Applied Sciences, Stralsund Mecklenburg - Vorpommern Wismar Schwerin Rostock Stralsund Greifswald Saßnitz Neubrandenburg Campus understanding reality facing challenges creating future 2

3 Unobstrusive fall detection and sensor systems interoperability for telemonitoring Contents: - Unobstrusive fall detection - Motivation and background - Concept and design - Validation with volunteers and in residential homes - Event processing - Summary - Sensor system interoperability for telemonitoring - Introduction and current status - CHA Guidelines for interoperability - Prototype implementation for assessment - Summary - Telemonitoring status in Germany understanding reality facing challenges creating future 3

4 Unobstrusive fall detection: Motivation fall statistics - incidence 30% for those over 65 years* - one fall per 6 months with persons in institutional care - 20% of falls lead to servere injuries and fractures - admission to hospitals - requiring continuous care - less independent living - costs for health and soical system - fall reasons - leaving the bed at night (no light, not fully wake, ) - hinderances, carpets, wheelchair, rollators - liability issues for institutional care - potential prevention - clarifying the fall situation - documenting the immedicate assistance after a fall *Center for Disease Control and Prevention, ) understanding reality facing challenges creating future 4

5 Fall detection approaches body worn sensor systems allowing to detect - acceleration - position - orientation - localisation - air pressure deviation pressure sensitive matts or floors movement detectors - infrared - ultrasound - radar camera systems - 2D - 3D but most approaches focus on the fall situation do not allow for prevention understanding reality facing challenges creating future 5

6 Fall detection using cameras advantages - fall documentation - support prevention by alerts (e.g. nurses) disadvantages - instrusive with regard to privacy - extensive image processing - costs MS-Kinect - re-use from mass-market (gaming) - 2D colour images - 3D sensor (640 x 480 pixel) infrared daylight independent - 3D microphone - low price (~100 ) - API available understanding reality facing challenges creating future 6

7 Fall detection concept unobstrusiveness - camera positioned on the floor - up to 50cm above the floor privacy - 3D image only - image processing locally targeted events - fall - leaving the bed / feet in front - leaving the room - activity in the room image analysis based on - width and hight of object - orientation - compactness - identified room regions understanding reality facing challenges creating future 7

8 Probability estimation situation: feet in front of the bed derived from 3D image no probability += 10% yes probalility += 20% two contours with an area difference up to maximal 30%? contours in the area in front of the bed? += 20% += 30% += 20% += 10% += 20% += 20% += 10% += 30% += 10% += 20% += 20% += 10% contour height >= 75% of distance bottom edge of the bed to floor? orientation of the contour line vertical? distance between contour less than 110cm? probablities: 100% feed in front of the bed 100% activity 20% fall understanding reality facing challenges creating future 8 C. Fukuoka, Marzahl MST, /

9 Prototype installation residential home Patientenb Ti ett Bettbe sc reich h Patienten bett Ti sc h Bad Ba d- ber eic h Tür - ber eich Stu hl Garde robe prototype system - easy installation - low configuration requirements (region identification) validation with volunteers - simulated scenarios - probability adjustment in decision tree understanding reality facing challenges creating future 9 C. Fukuoka, Marzahl MST, /

10 Validation with volunteers an example understanding reality facing challenges creating future 10

11 Validation with volunteers results: fall no fall sum fall detected fall not detected sum fall detection rate (47 of 49) 96% true positve rate (47 of 127): 37% false-positive-rate (80 of 127): 63% fall classification acceptable too many false positive alarms next steps validate in real-life situation reduce false positive alarms understanding reality facing challenges creating future 11

12 Validation in residential home Residents inclusion criteria: - age 65 years - resident institutional care in residential home - no major limitations in movement capability of resident - increased fall risk identified (e.g. due to fall assessment / previous falls) - added value for resident and care support - resident or person in charge fully informed, signed consent - 3 resident (66-87 years old, 1-5 falls within the last 12 months) - 2 residents in residential home - 1 person living in her own flat with frequent care support study: - maximum duration 2 months - full recording of data to allow for different evaluation approaches understanding reality facing challenges creating future 12

13 Validation with residents in residential home room for two residents (two beds) one aggreed to support validation) room with one resident private flat of one person understanding reality facing challenges creating future 13

14 Results from validation (1) care personal: - positive attitude, expected support of their work residents: - distrust/ refusal related to camera based system - 3D depth image inconceivable - fall detection is expected to lead to immediate support after a fall recommendation / experience: - education targeted for different end user groups - present 3D image to increase acceptance - no negative feedback during test, even more ideas for new use-cases - identify reason for a fall - peace of mind for residents - monitor and get to know resident`s behaviour - in-time / immediate alerts understanding reality facing challenges creating future 14

15 Results from validation (2) in theory - direct network access, internet connectivity per room - fast recruitment of residents - good health status of the resident throughout the validation (no drop out) in real-life - up-to-date residential home, but with VDSL only - residential home in start-up phase lots of new residents - detoriation of residents health status - high data volumes per week (2 TB within 2 months) - as expected from volunteer tests: high false-positive alarm rate no falls occured within the study duration understanding reality facing challenges creating future 15

16 Results from validation (3) further organisation and technical problems movement Kinect power supply unplugged rearranging furntiture mal function Kinect obstructed view (rollator) malfunction driver Kinect understanding reality facing challenges creating future 16

17 Adresssing high false-positive rate - rollators and wheelchairs obstructing view adapt image analysis, identify rollators/wheelchairs - multiple persons in the room e.g. when providing care obtain information from care personal alert system use typical schedule of care personal activity - insuffcient coverage of area in the room by only one Kinect use wireless door sensor to detect e.g. going to the bathroom improve existing image processing relying on single images only allow for rules taking previous events / states into account Measures taken door sensor for bathroom use of Complex Event Processing for evaluation understanding reality facing challenges creating future 17

18 Prototype system architecture (1) fall detection event, status, central processing alert alert personal 3D-images USB Kinect image processing event generation resident s room event type fall activity feet Fath exit description fall of a person activity in the room, e.g. by walking feet in front of the bed bath door opened room door opened understanding reality facing challenges creating future 18

19 Prototype system architecture (2) fall detection event, status, central processing alert alert personnal storing events event analysis event-db- central processing alert-db storing alerts / alarm configuration server room understanding reality facing challenges creating future 19

20 Prototype system architecture (3) fall detection event, status, central processing alert Alert personnal forwarding alert LAN/WLAN Smartphone LAN/WLAN central processing nurse workplace residential home understanding reality facing challenges creating future 20

21 Complex event processing (CEP) - Event Processing Network (EPN) using Event Processing Agents which are able to process high rate event stream - abstraction of events - sensor events (object detected, e.g. door closed) - context events (activity in the room, e.g. feet in front of the bed) - situational event (fall, leaving the room, prolonged stay in the bath) - NEsper-Framework* used for implementation complex events event stream a bstraction single events event stream * understanding reality facing challenges creating future 21

22 Architecture complex event processing sensors services messaging CEP interface Image manager sensor data simple events complex events with context event message understanding reality facing challenges creating future 22

23 Details of event processing network (EPN) understanding reality facing challenges creating future 23

24 Complex Event Processing configuration select a.source from pattern [every (a=activityevent -> timer:interval(1 sec) and BathOpenedEvent -> timer:interval(1 sec) and BathClosedEvent -> timer:interval(5 sec) and not ActivityEvent)] understanding reality facing challenges creating future 24

25 CEP für Ereignisauswertung - Ereignisausgabe understanding reality facing challenges creating future 25

26 Improvement achieved by CEP Szenario pure image based analysis complex event processing activity in the room knees on the floor sitting on the floor some fall alarms frequent fall alarms frequent fall alarms no fall alarms some fall alarms only if in front of the camera rollator permanent fall alarms only if in front of the camera and sitting on the rollator blanket out of the bed standing door area standing near door frequent fall alarms frequent fall alarms permanent fall alarms no fall alarms no fall alarms no fall alarms understanding reality facing challenges creating future 26

27 Summary: Unobstrusive fall detection results - valuable information from real-life validation - new problems - new use-cases (probably more realistic as with usual living labs) - CEP reduces false-positive alerts - further sensors needed to improve context todo - continued validation with residents - extend to more than one Kinect, e.g. for rooms with two residents - improve system stability understanding reality facing challenges creating future 27

28 Unobstrusive fall detection and sensor systems interoperability for telemonitoring Contents: - Unobstrusive fall detection - Motivation and background - Concept and design - Validation with volunteers and in residential homes - Event processing - Summary - Sensor system interoperability for telemonitoring - Introduction and current status - CHA Guidelines for interoperability - Prototype implementation for assessment - Summary - Telemonitoring status in Germany understanding reality facing challenges creating future 28

29 Telehealth: Telemonitoring scenarios: participants: information: implementation: - patient health status monitoring - improving patient compliance - support for patient convalescence - patient consultation ( peace of mind, emergencies) - medical device monitoring (ICD cardiodifibrillators) - telemedicine service centers, medical professionals patient relatives - vital signs - activity - surveillance - proprietory medical devices - company specific services - lack in use of (available) standards or profiles - limited integration with other provider systems - Personal Health Monitoring (PHM) - Ambient Assited Living (AAL) - domotics / facility management understanding reality facing challenges creating future 29

30 Current status: Sensor systems at home vendor PH/AAL Provider product line 1 product line 2 product line n * * * evaluatiom, feedback, coaching adaptation for each product line hinderance - vendor specific and proprietory (i) sensor system, (ii) communikation (iii) data formats/representation (iv) vendor services - vendor specific user interfaces for customers and providers Customer view: sensor system limitation/vendor, no plug & play PH / AAL Service view: high complexity, effort and costs *copyright pictures from: understanding reality facing challenges creating future 30

31 Continua Health Alliance Guideline: Assessment Continua Health Alliance (CHA) - mission*: Continua is dedicated to establishing a system of interoperable personal connected health solutions with the knowledge that extending those solutions into the home fosters independence, empowers individuals and provides the opportunity for truly personalized health and wellness management. - approach: - reference architecture for PHM and AAL - interface specification, plug & play, based on standards - from sensors to electronic health records patient environment (home, mobile) telemonitoring service Electronic Health Record Sensor-/Actor-system (Agent) Basestation (Manager) Monitoring server EHR PAN Interface PAN Interface WAN Interface WAN Interface HRN Interface HRN Interface * understanding reality facing challenges creating future 31

32 Continua Health Alliance Guideline: Assessment patient environment (home, mobile) PAN Interface PAN Interface WAN Interface WAN Interface telemonitoring service HRN Interface Electronic Health Record HRN Interface ISO/IEEE IHE PCD-01 HL7 ORU^R01 HL7 PHMR IHE XDR Bluetooth TCP/IP TCP/IP Open issues to be adressed: - guideline usability - interface functionality (plug & play, assignment, semantic, ) - Implementation support (tools, effort, ) prototype implementation understanding reality facing challenges creating future 32

33 PAN-Interface Concept process to establish plug&play / connection Sensor-/Actor-system (Agent) Glucometer : MDS Koerperfett: Numeric Gewicht: Numeric conecton request DIM unknown DIM accepted sensor data plus context Basisstation (Manager) Manager requires DIM (Domain information Model) to establish a connection to the sensor if DIM is unknown with the Manager it has to be provided by the Agent DIM is stored at the Manager Sensor data are represented according to the DIM understanding reality facing challenges creating future 33/17

34 PAN Interface sensor-/actorsystem (Agent) base station (Manager) application level (IEEE xx): Common Device Specification Pulse Oximeter Cardiovascular Fitness ISO/IEEE PHD Common Device Specification Blood Pressure Strength and Fitness Telemonitoring Thermometer Glucose Meter Independent Living Activity Hub <<refine>> Health & Fitness Independent Assisted Living Weighing Scale Insulin Pump Medication Monitor ISO/IEEE PHD Optimized Exchange Protocol Domain Information Model Service Model protocoll (IEEE ) Optimized Exchange Protocol provides plug & play for multiple device types Communication Model Transportprofile Bluetooth - HDP USB - PHDC transport and protocols: - Bluetooth Health Device Profile (HDP) - USB Personal Health Device Class (PHDC) understanding reality facing challenges creating future 34

35 PAN Interface Implementation Thermometer (µcontroller) - Bluetooth communication - PAN prototype for Agent and base station understanding reality facing challenges creating future 35

36 WAN-Interface Concept sensor data (Agent) base station (Manager) patient demographic data Monitoring-Server DIM sensor data DIM DIM DIM (von Waage) (von Waage) map HL7 V2.6 ORU^R01 (IHE PCD-01) sensor data, patient demographic data understanding reality facing challenges creating future 36/17

37 WAN-Interface base station (Manager) Monitoring-Server IHE Patient Care Device - 01 (PCD-01) HL7 V2.6 compliant to IHE PCD-01 unsolicited report (ORU^R01) application level: DIM compliant sensor data mapped to HL7 ORU^R01 message according to IHE PCD-01 semantic annotation IEEE and IEEE PHD transportprofile IHE IT Infrastructure TF Vol. 2 Appendix V (Webservice WS-I Basic Profile) application domain: Semantik according IEEE and IEEE Personal Health Device transport level (TCP/IP based): Web Service IHE ITI TF Vol.2 Appendix V (SOAP 1.2, WS-I BP* und WS-I BSP*) * WS-I BP Web Service Interoperability Basic Profile, WS-I BSP Web Service Basic Security Profile understanding reality facing challenges creating future 37

38 WAN Interface Implementation scale: MDS SystemType = MDC_DEVICE_SPEC_SCALE SystemId = FF:EE:DD:05:04:03:02:01 DevConfigId = 0x4000 (extended Cfg) body fat: Numeric Type = MDC_BODY_FAT UnitCode= MDC_DIM_PERCENT SimpleNuObservedValue= 12 AbsoluteTimeStamp = T:1831:0000 body weight: Numeric Type = MDC_MASS_BODY_ACTUAL UnitCode= MDC_DIM_KILO_G SimpleNuObservedValue= 72 AbsoluteTimeStamp = T17:31:0000 HL72.6 ORU^R01 mit IHE-PCD-01 MSH ^~ & FHS^080019FFFF4F6AC0^EUI-64 FHTMZ ORU^R01^ORU_R01 MSG00001 P 2.6 NE AL IHE PCD ORU-R ^HL7^Universal ID^HL7 PID 0815^^^FHTMZ^PI Doehring^Tom^^^^^L M Zur Schwedenschanze 2^^Stralsund^^1837^Germany^B OBR 1 ABC12345^FHS Basisstation^080019FFFF4F6AC0^EUI-64 DEF12345^FHS Basisstation^080019FFFF4F6AC0^EUI ^MDC_DEV_SPEC_PROFILE_SCALE^MDC OBX 1 NM ^MDC_MASS_BODY_ACTUAL^MDC ^MDC_DIM_KILO_G^MDC R ^^FFEEDD ^EUI-64 OBX 2 NM ^MDC_BODY_FAT^MDC ^MDC_DIM_PERCENT^MDC R ^^FF EEDD ^EUI-64 understanding reality facing challenges creating future 38

39 HRN Interface Concept telemonitoring centre base station (Manager) sensor data, patient demographic data mapping sensor data HL7 ORU^R01 procedres Treatment protocol contact person Personal Health Monitoring Report (PHMR) attachments EHR / PHR report, patienten demographic data, sensor data understanding reality facing challenges creating future 39/17

40 HRN-Interface Monitoring-Server EHR HL7 Personal Healthcare Monitoring Report (PHMR) HL7 Implementation Guide for CDA R2 application level: mapping of sensor data (including DIM) to persistent CDA R2 Document (HL7 PHMR compliant) semantic annotation SNOMED, UCUM, LOINC, IEEE bzw. IEEE PHD application domain: SNOMED, UCUM, IEEE transport profiles IHE XDR IHE Cross Enterprise Document Reliable Interchange transport profile (TCP/IP) IHE XDR (HTTP/SOAP) understanding reality facing challenges creating future 40

41 HRN-Interface implementation: Tool Integrationsserver (EAI*) business process: mapping HL7-ORU message to CDA R2 document mapping: segment fields of the HL7-ORU message mapped to tags of the CDA R2 document, potentially with changing the representation and adding e.g. title, effectivetime, *EAI Enterprise Application Integration understanding reality facing challenges creating future 41

42 HRN-Interface implementation: Tool HL72.6 ORU^R01 using IHE-PCD-01 MSH ^~ & FHS^080019FFFF4F6AC0^EUI-64 FHTMZ ORU^R01^ORU_R01 MSG00001 P 2.6 NE AL IHE PCD ORU-R ^HL7^Universal ID^HL7 PID 0815^^^FHTMZ^PI Doehring^Tom^^^^^L M Zur Schwedenschanze 2^^Stralsund^^1837^Germany^B OBR 1 ABC12345^FHS Basisstation^080019FFFF4F6AC0^EUI-64 DEF12345^FHS Basisstation^080019FFFF4F6AC0^EUI ^MDC_DEV_SPEC_PROFILE_SCALE^MDC OBX 1 NM ^MDC_MASS_BODY_ACTUAL^MDC ^MDC_DIM_KILO_G^MDC R ^^FFEEDD ^EUI-64 <component> sensor data <observation classcode="obs" moodcode="evn"> OBX 1 NM ^MDC_BODY_FAT^MDC ^MDC_DIM_PERCENT^MDC R ^^FF EEDD ^EUI-64 (units according to UCUM) sensor device code system <templateid root=" "/> <templateid root=" "/> <id root="975c2f3b-2bd4-4e45-aed1-84af9ff51b10"/> <code code="mdc_mass_body_actual" codesystem=" " codesystemname="mdc" displayname="mass Body Actual"/> <statuscode code="completed"/> <effectivetime value=" "/> <value xsi:type="pq" value="72" unit="kg"/> <participant typecode="dev"> <participantrole> <id root=" " assigningauthorityname="eui-64" extension="ff-ee-dd "/> </participantrole> </participant> understanding reality facing challenges creating future 42

43 Assessment results - prototyp implementation confirmed - standard compliant communication sensor EHR - plug & play at the PAN Schnittstelle - semantic annotation of sensor data based on the DIM - acceptable implementation effort - encryption support on data transport level - context information can be added as needed at each level Patient Environment (home, mobile) Telemonitoring Centre Electronic Health Record sensor-/actor-system (Agent) measurement data, -context base station (Manager) caring, coaching of the patient monitoring server events, measures, report EHR understanding reality facing challenges creating future 43

44 Continua Health Alliance Guideline: Validation and assessment patient environment (home, mobile) PAN Interface PAN Interface WAN Interface WAN Interface telemonitoring service HRN Interface Electronic Health Record HRN Interface ISO/IEEE IHE PCD-01 HL7 ORU^R01 HL7 PHMR IHE XDR Bluetooth TCP/IP TCP/IP open issues: - bi-directional communication with sensor - pairing sensor system user identifiction at which level (Manager, TM Centre via device ID) - domain model limitations for event (e.g. fall) - extending to further transfer protocols (e.g. ANT+) - mutual authentication at PAN-level CHA well suited to establish cross-vendor interoperability CHA guidelines support customer and provider requirements understanding reality facing challenges creating future 44

45 Telemonitoring in Germany medical device monitoring (e.g. ICDs) - well established, costs covered by health insurance - reduction of visits to cardiological out-patient sevices patient health status monitoring - multiple projects by health insurance companies to assess and provide evidence for monitoring, coaching, adherance, and surveillance programmes - some service and infrastructure providers - may be contracted by patients/citiziens at own costs - no reimbursement by health insurances but - on-going check (till April 2013) for medical service provision using telehealth approaches status 2013: - sensors, communication, services available - limited interoperability - lack of evidence for specific medical scenarios - missing reimbursement for wider use understanding reality facing challenges creating future 45

46 Unobstrusive fall detection and sensor systems interoperability thanks for your attention? questions? Acknowledgement: Unobstrusive fall detection: CHA prototype implementation: Christian Marzahl, Peter Penndorf, Henriette Rau, Jacob Grieger Tom Döhring, understanding reality facing challenges creating future 46