Copyright 2013 Augmented Vision - DFKI 06.12.2013 1 Visual and mobile Smart Data Didier Stricker didier.stricker@dfki.de
Department Augmented Vision @ DFKI Head: Didier Stricker Founded in July 2008 30 fulltime researchers 3 strongly connected research areas Computer Vision & Video Analytics Augmented Reality, Visualization & HCI Body Sensor Networks & Sensor Interpretation 2
3 Big Data Taxonomy IDC. IDC s Worldwide Big Data Taxonomy,2011. [Online] Available from: http://www.idc.com/getdoc.jsp?containerid=23 1099 [Accessed 9th July 2013].
Outline Big Visual Data 1. New Media (hyper videos, video archive, ) 2. Security tracking and investigation 3. E-Commerce 4. 3D scene reconstruction Scalability of current solutions? To develop novel image/video understanding approaches assuming Big (Visual) Data? User Physical Activity Monitoring and Geo-Information-System (GIS) 1. Complexity and personalization 2. Streaming and visualization into GIS Scalability of streaming solution GIS and Big (Sensor) Data 4
Outline Big Visual Data 1. New Media (hyper videos, video archive, ) 2. Security tracking and investigation 3. E-Commerce 4. 3D scene reconstruction Scalability of current solutions? To develop novel image/video understanding approaches assuming Big (Visual) Data? User Physical Activity Monitoring and Geo-Information-System (GIS) 1. Complexity and personalization 2. Streaming and visualization into GIS Scalability of streaming solution GIS and Big (Sensor) Data 5
Object detection/retrieval Reference region in one image???
Object detection Problem Full exhaustive search of the best region in the complete image at different scales. Example: 70.000 regions at 19 scales for an image 320x240: 15 secondes New solution: Integral images + new region descriptor -7-
Exhaustive search -8-
Computation time -9-
Error in pixel to the Ground Truth region Images de 320x240-10-
Tracking
13 Hypervideos Adding informations to objects in videos Text, images, hyperlinks etc.
Images in E-commerce Images are required for E-Commerce Printing media Shop planning Required Functions Finding / retrieving images Image comparison Extracting meta-data Ingredient 21/11/2013 15
Scalable solution for extraction of text in natural images Input image Detected text area Top and bottom line Normalized text area OCR Peacocks
Large scale modeling High-quality camera 100 Million Pixels Spherical images High Dynamic Range An image is one Gigabyte large 18
Towards Giga-Pixel image-processing 19
20
3D Reconstruction from spherical images
Fritz-Walter-Stadion Calibrated cameras 76 Sparse points 803,231 Reconstructed points Measures 240,101,306 215m x 170 m Input images Structure From Motion Dense pointcloud
Fritz-Walter-Stadium
Pagani et al.: Dense 3D point cloud generation from multiple high resolution spherical images
Outline Big Visual Data Large image and 3D reconstruction Video Content Analytics User Activity Monitoring and Geo-Information-System (GIS) Activity recognition: complexity and personalization Streaming and visualization into GIS EASY Kick-Off Meeting 21/11/2013 27
DFKI 12/6/2013 28 Two different physical activity (PA) categories aerobic or endurance To promote cardio-vascular health strength or stretching To improve or maintain strength/balance Intensity Duration Type X Intensity Technique Quality Global activity profile Monitoring over active time of the day Reduced sensor setup Exercise prescription compliance Monitoring over one exercise session (< 1 h) Full body motion tracking!
(Personalized) physical activity monitoring 21/11/2013 29
Everyday life activity monitoring Estimating intensity of general activities, 15 activities assigned to 3 intensity classes: Light intensity activities (< 3.0 METs) lie, sit, stand, drive car, iron, fold laundry, clean house, watch TV, computer work Moderate intensity activities (3.0 6.0 METs) walk, cycle, descend stairs, vacuum clean, Nordic walk Vigorous intensity activities (> 6.0 METs) run, ascend stairs, rope jump, play soccer This exceeds the potential of existing classifiers Aminian 2004
ISWC 2013 31 ConfAdaBoost.M1 III the more confident the weak learner is in an instance's correct classification / misclassification the more that instance's weight is reduced / increased weak learner returns the confidence of the classification estimation the more confident the weak learner is in a new instance's prediction, the more it counts in the final combined classifier
32 A new approach Test on the UCI database Diversity in how subjects perform physical activities personalization
21/11/2013 33 Personalization Basic idea: Everyone is different! Record your own data for a given activity over only one minute 1 2 3 123 Summary Other Lying Sitting Standing Record about 6 out of 15 activities Alerts Upload Walking Running Cycling Nordic walking Re-arrangement of the classifiers (on the phone) Improve your recognition score
Cloud Service (e.g. SOS) Visualize data with GeoVisualizer Smart phone to track internal and external data / information
12/6/2013 36 TrackMe Overview Tracks, logs, and uploads (offline and online) of data from sensors (internal and external) Acts as a data logger, records data, and calculates (low and high-level) information even when not connected to the internet / cloud Retrieves data via Bluetooth from external sensors Retrieves data from available internal sensors, e.g. accelerometer, GPS, Gyro, Light, Audio Computing the following low and high-level information Push data manually to the cloud from within the TrackMe app or automatically upload for real-time monitoring
GeoVisualizer Overview (cont.) Developed in the European project SUDPLAN (www.sudplan.eu) Different end-user requirements (e.g. data format, visualization, color mapping,...) Easy-to-use toolkit Licensed as open-source application under the LGPL version 3.0 Latest stable release 3.2.0 Air-Quality Results Iso-Surface Visualization Visualization of Bio-Feedback 40
Continuous real-time analysis and visualisation SUDPLAN Sustainable Urban Development Planner for Climate Change Adaptation
Outlook Current focus of research Video / image analysis: large scale application Actvitiy monitoring: long term autonomy Out interest in this workshop Learn from experience in other field (bio-informatics, ) Transfer our approaches to other imaging or data modalities (bio-medicine,..)
Copyright 2013 Augmented Vision - DFKI 06.12.2013 43 Thank you for your attention! DFKI GmbH Department Augmented Vision T rippstadterst r. 122 D-67663 K aiserslautern Didier Stricker Didier.stricker@dfki.de http://av.dfki.de/