Health for All - Nationally And Internationally by Medical Informatics Unobtrusive fall detection and sensor system interoperability for telemonitoring Martin Staemmler Medical Informatics, University of Applied Sciences, Stralsund contact: martin.staemmler@fh-.de understanding reality facing challenges creating future 1
Unversity of Applied Sciences, Stralsund Mecklenburg - Vorpommern Wismar Schwerin Rostock Stralsund Greifswald Saßnitz Neubrandenburg Campus understanding reality facing challenges creating future 2
Unobstrusive fall detection and sensor systems interoperability for telemonitoring Contents: - Unobstrusive fall detection - Motivation and background - Concept and design - Validation with volunteers and in residential homes - Event processing - Summary - Sensor system interoperability for telemonitoring - Introduction and current status - CHA Guidelines for interoperability - Prototype implementation for assessment - Summary - Telemonitoring status in Germany understanding reality facing challenges creating future 3
Unobstrusive fall detection: Motivation fall statistics - incidence 30% for those over 65 years* - one fall per 6 months with persons in institutional care - 20% of falls lead to servere injuries and fractures - admission to hospitals - requiring continuous care - less independent living - costs for health and soical system - fall reasons - leaving the bed at night (no light, not fully wake, ) - hinderances, carpets, wheelchair, rollators - liability issues for institutional care - potential prevention - clarifying the fall situation - documenting the immedicate assistance after a fall *Center for Disease Control and Prevention,www.cdc.gov/HomeandRecreationlSafty/adultfalls.html(Stand 20.09.2011) understanding reality facing challenges creating future 4
Fall detection approaches body worn sensor systems allowing to detect - acceleration - position - orientation - localisation - air pressure deviation pressure sensitive matts or floors movement detectors - infrared - ultrasound - radar camera systems - 2D - 3D but most approaches focus on the fall situation do not allow for prevention understanding reality facing challenges creating future 5
Fall detection using cameras advantages - fall documentation - support prevention by alerts (e.g. nurses) disadvantages - instrusive with regard to privacy - extensive image processing - costs MS-Kinect - re-use from mass-market (gaming) - 2D colour images - 3D sensor (640 x 480 pixel) infrared daylight independent - 3D microphone - low price (~100 ) - API available understanding reality facing challenges creating future 6
Fall detection concept unobstrusiveness - camera positioned on the floor - up to 50cm above the floor privacy - 3D image only - image processing locally targeted events - fall - leaving the bed / feet in front - leaving the room - activity in the room image analysis based on - width and hight of object - orientation - compactness - identified room regions understanding reality facing challenges creating future 7
Probability estimation situation: feet in front of the bed derived from 3D image no probability += 10% yes probalility += 20% two contours with an area difference up to maximal 30%? contours in the area in front of the bed? += 20% += 30% += 20% += 10% += 20% += 20% += 10% += 30% += 10% += 20% += 20% += 10% contour height >= 75% of distance bottom edge of the bed to floor? orientation of the contour line vertical? distance between contour less than 110cm? probablities: 100% feed in front of the bed 100% activity 20% fall understanding reality facing challenges creating future 8 C. Fukuoka, Marzahl MST, 18.3.2013 8 /
Prototype installation residential home Patientenb Ti ett Bettbe sc reich h Patienten bett Ti sc h Bad Ba d- ber eic h Tür - ber eich Stu hl Garde robe prototype system - easy installation - low configuration requirements (region identification) validation with volunteers - simulated scenarios - probability adjustment in decision tree understanding reality facing challenges creating future 9 C. Fukuoka, Marzahl MST, 18.3.2013 9 /
Validation with volunteers an example understanding reality facing challenges creating future 10
Validation with volunteers results: fall no fall sum fall detected 47 80 127 fall not detected 2 219 221 sum 49 299 348 fall detection rate (47 of 49) 96% true positve rate (47 of 127): 37% false-positive-rate (80 of 127): 63% fall classification acceptable too many false positive alarms next steps validate in real-life situation reduce false positive alarms understanding reality facing challenges creating future 11
Validation in residential home Residents inclusion criteria: - age 65 years - resident institutional care in residential home - no major limitations in movement capability of resident - increased fall risk identified (e.g. due to fall assessment / previous falls) - added value for resident and care support - resident or person in charge fully informed, signed consent - 3 resident (66-87 years old, 1-5 falls within the last 12 months) - 2 residents in residential home - 1 person living in her own flat with frequent care support study: - maximum duration 2 months - full recording of data to allow for different evaluation approaches understanding reality facing challenges creating future 12
Validation with residents in residential home room for two residents (two beds) one aggreed to support validation) room with one resident private flat of one person understanding reality facing challenges creating future 13
Results from validation (1) care personal: - positive attitude, expected support of their work residents: - distrust/ refusal related to camera based system - 3D depth image inconceivable - fall detection is expected to lead to immediate support after a fall recommendation / experience: - education targeted for different end user groups - present 3D image to increase acceptance - no negative feedback during test, even more ideas for new use-cases - identify reason for a fall - peace of mind for residents - monitor and get to know resident`s behaviour - in-time / immediate alerts understanding reality facing challenges creating future 14
Results from validation (2) in theory - direct network access, internet connectivity per room - fast recruitment of residents - good health status of the resident throughout the validation (no drop out) in real-life - up-to-date residential home, but with VDSL only - residential home in start-up phase lots of new residents - detoriation of residents health status - high data volumes per week (2 TB within 2 months) - as expected from volunteer tests: high false-positive alarm rate no falls occured within the study duration understanding reality facing challenges creating future 15
Results from validation (3) further organisation and technical problems movement Kinect power supply unplugged rearranging furntiture mal function Kinect obstructed view (rollator) malfunction driver Kinect understanding reality facing challenges creating future 16
Adresssing high false-positive rate - rollators and wheelchairs obstructing view adapt image analysis, identify rollators/wheelchairs - multiple persons in the room e.g. when providing care obtain information from care personal alert system use typical schedule of care personal activity - insuffcient coverage of area in the room by only one Kinect use wireless door sensor to detect e.g. going to the bathroom improve existing image processing relying on single images only allow for rules taking previous events / states into account Measures taken door sensor for bathroom use of Complex Event Processing for evaluation understanding reality facing challenges creating future 17
Prototype system architecture (1) fall detection event, status, central processing alert alert personal 3D-images USB Kinect image processing event generation resident s room event type fall activity feet Fath exit description fall of a person activity in the room, e.g. by walking feet in front of the bed bath door opened room door opened understanding reality facing challenges creating future 18
Prototype system architecture (2) fall detection event, status, central processing alert alert personnal storing events event analysis event-db- central processing alert-db storing alerts / alarm configuration server room understanding reality facing challenges creating future 19
Prototype system architecture (3) fall detection event, status, central processing alert Alert personnal forwarding alert LAN/WLAN Smartphone LAN/WLAN central processing nurse workplace residential home understanding reality facing challenges creating future 20
Complex event processing (CEP) - Event Processing Network (EPN) using Event Processing Agents which are able to process high rate event stream - abstraction of events - sensor events (object detected, e.g. door closed) - context events (activity in the room, e.g. feet in front of the bed) - situational event (fall, leaving the room, prolonged stay in the bath) - NEsper-Framework* used for implementation complex events event stream a bstraction single events event stream * http://esper.codehaus.org/about/nesper/nesper.html understanding reality facing challenges creating future 21
Architecture complex event processing sensors services messaging CEP interface Image manager sensor data simple events complex events with context event message understanding reality facing challenges creating future 22
Details of event processing network (EPN) understanding reality facing challenges creating future 23
Complex Event Processing configuration select a.source from pattern [every (a=activityevent -> timer:interval(1 sec) and BathOpenedEvent -> timer:interval(1 sec) and BathClosedEvent -> timer:interval(5 sec) and not ActivityEvent)] understanding reality facing challenges creating future 24
CEP für Ereignisauswertung - Ereignisausgabe understanding reality facing challenges creating future 25
Improvement achieved by CEP Szenario pure image based analysis complex event processing activity in the room knees on the floor sitting on the floor some fall alarms frequent fall alarms frequent fall alarms no fall alarms some fall alarms only if in front of the camera rollator permanent fall alarms only if in front of the camera and sitting on the rollator blanket out of the bed standing door area standing near door frequent fall alarms frequent fall alarms permanent fall alarms no fall alarms no fall alarms no fall alarms understanding reality facing challenges creating future 26
Summary: Unobstrusive fall detection results - valuable information from real-life validation - new problems - new use-cases (probably more realistic as with usual living labs) - CEP reduces false-positive alerts - further sensors needed to improve context todo - continued validation with residents - extend to more than one Kinect, e.g. for rooms with two residents - improve system stability understanding reality facing challenges creating future 27
Unobstrusive fall detection and sensor systems interoperability for telemonitoring Contents: - Unobstrusive fall detection - Motivation and background - Concept and design - Validation with volunteers and in residential homes - Event processing - Summary - Sensor system interoperability for telemonitoring - Introduction and current status - CHA Guidelines for interoperability - Prototype implementation for assessment - Summary - Telemonitoring status in Germany understanding reality facing challenges creating future 28
Telehealth: Telemonitoring scenarios: participants: information: implementation: - patient health status monitoring - improving patient compliance - support for patient convalescence - patient consultation ( peace of mind, emergencies) - medical device monitoring (ICD cardiodifibrillators) - telemedicine service centers, medical professionals patient relatives - vital signs - activity - surveillance - proprietory medical devices - company specific services - lack in use of (available) standards or profiles - limited integration with other provider systems - Personal Health Monitoring (PHM) - Ambient Assited Living (AAL) - domotics / facility management understanding reality facing challenges creating future 29
Current status: Sensor systems at home vendor PH/AAL Provider product line 1 product line 2 product line n * * * evaluatiom, feedback, coaching adaptation for each product line hinderance - vendor specific and proprietory (i) sensor system, (ii) communikation (iii) data formats/representation (iv) vendor services - vendor specific user interfaces for customers and providers Customer view: sensor system limitation/vendor, no plug & play PH / AAL Service view: high complexity, effort and costs *copyright pictures from: www.aipermon.com/produkte.html, www.fonium.de/fonium/index.html, www.smartlab.org, www.hmm.info understanding reality facing challenges creating future 30
Continua Health Alliance Guideline: Assessment Continua Health Alliance (CHA) - mission*: Continua is dedicated to establishing a system of interoperable personal connected health solutions with the knowledge that extending those solutions into the home fosters independence, empowers individuals and provides the opportunity for truly personalized health and wellness management. - approach: - reference architecture for PHM and AAL - interface specification, plug & play, based on standards - from sensors to electronic health records patient environment (home, mobile) telemonitoring service Electronic Health Record Sensor-/Actor-system (Agent) Basestation (Manager) Monitoring server EHR PAN Interface PAN Interface WAN Interface WAN Interface HRN Interface HRN Interface *www.continuaalliance.org understanding reality facing challenges creating future 31
Continua Health Alliance Guideline: Assessment patient environment (home, mobile) PAN Interface PAN Interface WAN Interface WAN Interface telemonitoring service HRN Interface Electronic Health Record HRN Interface ISO/IEEE 11073-20601 IHE PCD-01 HL7 ORU^R01 HL7 PHMR IHE XDR Bluetooth TCP/IP TCP/IP Open issues to be adressed: - guideline usability - interface functionality (plug & play, assignment, semantic, ) - Implementation support (tools, effort, ) prototype implementation understanding reality facing challenges creating future 32
PAN-Interface Concept process to establish plug&play / connection Sensor-/Actor-system (Agent) Glucometer : MDS Koerperfett: Numeric Gewicht: Numeric conecton request DIM unknown DIM accepted sensor data plus context Basisstation (Manager) Manager requires DIM (Domain information Model) to establish a connection to the sensor if DIM is unknown with the Manager it has to be provided by the Agent DIM is stored at the Manager Sensor data are represented according to the DIM understanding reality facing challenges creating future 33/17
PAN Interface sensor-/actorsystem (Agent) base station (Manager) application level (IEEE 11073-104xx): Common Device Specification 10404 Pulse Oximeter 10441 Cardiovascular Fitness ISO/IEEE 11073 PHD - 10400 Common Device Specification 10407 Blood Pressure 10442 Strength and Fitness Telemonitoring 10408 Thermometer 10417 Glucose Meter 10471 Independent Living Activity Hub <<refine>> Health & Fitness Independent Assisted Living 10418 Weighing Scale 10419 Insulin Pump 10472 Medication Monitor ISO/IEEE 11073 PHD - 20601 Optimized Exchange Protocol Domain Information Model Service Model protocoll (IEEE 11073-20601) Optimized Exchange Protocol provides plug & play for multiple device types Communication Model Transportprofile Bluetooth - HDP USB - PHDC transport and protocols: - Bluetooth Health Device Profile (HDP) - USB Personal Health Device Class (PHDC) understanding reality facing challenges creating future 34
PAN Interface Implementation - 10408 Thermometer (µcontroller) - Bluetooth communication - PAN prototype for Agent and base station understanding reality facing challenges creating future 35
WAN-Interface Concept sensor data (Agent) base station (Manager) patient demographic data Monitoring-Server DIM sensor data 11073-20601 DIM DIM DIM (von Waage) (von Waage) map HL7 V2.6 ORU^R01 (IHE PCD-01) sensor data, patient demographic data understanding reality facing challenges creating future 36/17
WAN-Interface base station (Manager) Monitoring-Server IHE Patient Care Device - 01 (PCD-01) HL7 V2.6 compliant to IHE PCD-01 unsolicited report (ORU^R01) application level: DIM compliant sensor data mapped to HL7 ORU^R01 message according to IHE PCD-01 semantic annotation IEEE 11073-10101 and IEEE 11073-PHD transportprofile IHE IT Infrastructure TF Vol. 2 Appendix V (Webservice WS-I Basic Profile) application domain: Semantik according IEEE 11073-10101 and IEEE 11073-Personal Health Device transport level (TCP/IP based): Web Service IHE ITI TF Vol.2 Appendix V (SOAP 1.2, WS-I BP* und WS-I BSP*) * WS-I BP Web Service Interoperability Basic Profile, WS-I BSP Web Service Basic Security Profile understanding reality facing challenges creating future 37
WAN Interface Implementation scale: MDS SystemType = MDC_DEVICE_SPEC_SCALE SystemId = FF:EE:DD:05:04:03:02:01 DevConfigId = 0x4000 (extended Cfg) body fat: Numeric Type = MDC_BODY_FAT UnitCode= MDC_DIM_PERCENT SimpleNuObservedValue= 12 AbsoluteTimeStamp = 2010-11-03T:1831:0000 body weight: Numeric Type = MDC_MASS_BODY_ACTUAL UnitCode= MDC_DIM_KILO_G SimpleNuObservedValue= 72 AbsoluteTimeStamp = 2010-11-03T17:31:0000 HL72.6 ORU^R01 mit IHE-PCD-01 MSH ^~ & FHS^080019FFFF4F6AC0^EUI-64 FHTMZ 20101103173021 ORU^R01^ORU_R01 MSG00001 P 2.6 NE AL IHE PCD ORU-R01 2010^HL7^Universal ID^HL7 PID 0815^^^FHTMZ^PI Doehring^Tom^^^^^L M Zur Schwedenschanze 2^^Stralsund^^1837^Germany^B OBR 1 ABC12345^FHS Basisstation^080019FFFF4F6AC0^EUI-64 DEF12345^FHS Basisstation^080019FFFF4F6AC0^EUI- 64 528399^MDC_DEV_SPEC_PROFILE_SCALE^MDC 20101103000001+0000 20101103235959+0000 OBX 1 NM 188736^MDC_MASS_BODY_ACTUAL^MDC 1.0.0.1 72 263875^MDC_DIM_KILO_G^MDC R 20101103173121+ 0000 ^^FFEEDD0504030201^EUI-64 OBX 2 NM 188748^MDC_BODY_FAT^MDC 1.0.0.2 12 262688^MDC_DIM_PERCENT^MDC R 20101103173121+0000 ^^FF EEDD0504030201^EUI-64 understanding reality facing challenges creating future 38
HRN Interface Concept telemonitoring centre base station (Manager) sensor data, patient demographic data mapping sensor data HL7 ORU^R01 procedres Treatment protocol contact person Personal Health Monitoring Report (PHMR) attachments EHR / PHR report, patienten demographic data, sensor data understanding reality facing challenges creating future 39/17
HRN-Interface Monitoring-Server EHR HL7 Personal Healthcare Monitoring Report (PHMR) HL7 Implementation Guide for CDA R2 application level: mapping of sensor data (including DIM) to persistent CDA R2 Document (HL7 PHMR compliant) semantic annotation SNOMED, UCUM, LOINC, IEEE 11073-10101 bzw. IEEE 11073-PHD application domain: SNOMED, UCUM, IEEE 11073 transport profiles IHE XDR IHE Cross Enterprise Document Reliable Interchange transport profile (TCP/IP) IHE XDR (HTTP/SOAP) understanding reality facing challenges creating future 40
HRN-Interface implementation: Tool Integrationsserver (EAI*) business process: mapping HL7-ORU message to CDA R2 document mapping: segment fields of the HL7-ORU message mapped to tags of the CDA R2 document, potentially with changing the representation and adding e.g. title, effectivetime, *EAI Enterprise Application Integration understanding reality facing challenges creating future 41
HRN-Interface implementation: Tool HL72.6 ORU^R01 using IHE-PCD-01 MSH ^~ & FHS^080019FFFF4F6AC0^EUI-64 FHTMZ 20101103173021 ORU^R01^ORU_R01 MSG00001 P 2.6 NE AL IHE PCD ORU-R01 2010^HL7^Universal ID^HL7 PID 0815^^^FHTMZ^PI Doehring^Tom^^^^^L M Zur Schwedenschanze 2^^Stralsund^^1837^Germany^B OBR 1 ABC12345^FHS Basisstation^080019FFFF4F6AC0^EUI-64 DEF12345^FHS Basisstation^080019FFFF4F6AC0^EUI- 64 528399^MDC_DEV_SPEC_PROFILE_SCALE^MDC 20101103000001+0000 20101103235959+0000 OBX 1 NM 188736^MDC_MASS_BODY_ACTUAL^MDC 1.0.0.1 72 263875^MDC_DIM_KILO_G^MDC R 20101103173121+ 0000 ^^FFEEDD0504030201^EUI-64 <component> sensor data <observation classcode="obs" moodcode="evn"> OBX 1 NM 188748^MDC_BODY_FAT^MDC 1.0.0.2 12 262688^MDC_DIM_PERCENT^MDC R 20101103173121+0000 ^^FF EEDD0504030201^EUI-64 (units according to UCUM) sensor device code system <templateid root="2.16.840.1.113883.10.20.1.31"/> <templateid root="2.16.840.1.113883.10.20.9.8"/> <id root="975c2f3b-2bd4-4e45-aed1-84af9ff51b10"/> <code code="mdc_mass_body_actual" codesystem="2.16.840.1.113883.6.24" codesystemname="mdc" displayname="mass Body Actual"/> <statuscode code="completed"/> <effectivetime value="20101103173121"/> <value xsi:type="pq" value="72" unit="kg"/> <participant typecode="dev"> <participantrole> <id root="1.2.840.10004.1.1.1.0.0.1.0.0.1.2680" assigningauthorityname="eui-64" extension="ff-ee-dd-05-04-03-02-01"/> </participantrole> </participant> understanding reality facing challenges creating future 42
Assessment results - prototyp implementation confirmed - standard compliant communication sensor EHR - plug & play at the PAN Schnittstelle - semantic annotation of sensor data based on the DIM - acceptable implementation effort - encryption support on data transport level - context information can be added as needed at each level Patient Environment (home, mobile) Telemonitoring Centre Electronic Health Record sensor-/actor-system (Agent) measurement data, -context base station (Manager) caring, coaching of the patient monitoring server events, measures, report EHR understanding reality facing challenges creating future 43
Continua Health Alliance Guideline: Validation and assessment patient environment (home, mobile) PAN Interface PAN Interface WAN Interface WAN Interface telemonitoring service HRN Interface Electronic Health Record HRN Interface ISO/IEEE 11073-20601 IHE PCD-01 HL7 ORU^R01 HL7 PHMR IHE XDR Bluetooth TCP/IP TCP/IP open issues: - bi-directional communication with sensor - pairing sensor system user identifiction at which level (Manager, TM Centre via device ID) - domain model limitations for event (e.g. fall) - extending to further transfer protocols (e.g. ANT+) - mutual authentication at PAN-level CHA well suited to establish cross-vendor interoperability CHA guidelines support customer and provider requirements understanding reality facing challenges creating future 44
Telemonitoring in Germany medical device monitoring (e.g. ICDs) - well established, costs covered by health insurance - reduction of visits to cardiological out-patient sevices patient health status monitoring - multiple projects by health insurance companies to assess and provide evidence for monitoring, coaching, adherance, and surveillance programmes - some service and infrastructure providers - may be contracted by patients/citiziens at own costs - no reimbursement by health insurances but - on-going check (till April 2013) for medical service provision using telehealth approaches status 2013: - sensors, communication, services available - limited interoperability - lack of evidence for specific medical scenarios - missing reimbursement for wider use understanding reality facing challenges creating future 45
Unobstrusive fall detection and sensor systems interoperability thanks for your attention? questions? Acknowledgement: Unobstrusive fall detection: CHA prototype implementation: Christian Marzahl, Peter Penndorf, Henriette Rau, Jacob Grieger Tom Döhring, understanding reality facing challenges creating future 46