Visual and mobile Smart Data

Transcription

1 Copyright 2013 Augmented Vision - DFKI Visual and mobile Smart Data Didier Stricker [email protected]

2 Department Augmented DFKI Head: Didier Stricker Founded in July fulltime researchers 3 strongly connected research areas Computer Vision & Video Analytics Augmented Reality, Visualization & HCI Body Sensor Networks & Sensor Interpretation 2

3 3 Big Data Taxonomy IDC. IDC s Worldwide Big Data Taxonomy,2011. [Online] Available from: [Accessed 9th July 2013].

4 Outline Big Visual Data 1. New Media (hyper videos, video archive, ) 2. Security tracking and investigation 3. E-Commerce 4. 3D scene reconstruction Scalability of current solutions? To develop novel image/video understanding approaches assuming Big (Visual) Data? User Physical Activity Monitoring and Geo-Information-System (GIS) 1. Complexity and personalization 2. Streaming and visualization into GIS Scalability of streaming solution GIS and Big (Sensor) Data 4

5 Outline Big Visual Data 1. New Media (hyper videos, video archive, ) 2. Security tracking and investigation 3. E-Commerce 4. 3D scene reconstruction Scalability of current solutions? To develop novel image/video understanding approaches assuming Big (Visual) Data? User Physical Activity Monitoring and Geo-Information-System (GIS) 1. Complexity and personalization 2. Streaming and visualization into GIS Scalability of streaming solution GIS and Big (Sensor) Data 5

6 Object detection/retrieval Reference region in one image???

7 Object detection Problem Full exhaustive search of the best region in the complete image at different scales. Example: regions at 19 scales for an image 320x240: 15 secondes New solution: Integral images + new region descriptor -7-

8 Exhaustive search -8-

9 Computation time -9-

10 Error in pixel to the Ground Truth region Images de 320x

11 Tracking

12 13 Hypervideos Adding informations to objects in videos Text, images, hyperlinks etc.

13

14 Images in E-commerce Images are required for E-Commerce Printing media Shop planning Required Functions Finding / retrieving images Image comparison Extracting meta-data Ingredient 21/11/

15 Scalable solution for extraction of text in natural images Input image Detected text area Top and bottom line Normalized text area OCR Peacocks

16 Large scale modeling High-quality camera 100 Million Pixels Spherical images High Dynamic Range An image is one Gigabyte large 18

17 Towards Giga-Pixel image-processing 19

18 20

19 3D Reconstruction from spherical images

20 Fritz-Walter-Stadion Calibrated cameras 76 Sparse points 803,231 Reconstructed points Measures 240,101, m x 170 m Input images Structure From Motion Dense pointcloud

21

22

23 Fritz-Walter-Stadium

24 Pagani et al.: Dense 3D point cloud generation from multiple high resolution spherical images

25 Outline Big Visual Data Large image and 3D reconstruction Video Content Analytics User Activity Monitoring and Geo-Information-System (GIS) Activity recognition: complexity and personalization Streaming and visualization into GIS EASY Kick-Off Meeting 21/11/

26 DFKI 12/6/ Two different physical activity (PA) categories aerobic or endurance To promote cardio-vascular health strength or stretching To improve or maintain strength/balance Intensity Duration Type X Intensity Technique Quality Global activity profile Monitoring over active time of the day Reduced sensor setup Exercise prescription compliance Monitoring over one exercise session (< 1 h) Full body motion tracking!

27 (Personalized) physical activity monitoring 21/11/

28 Everyday life activity monitoring Estimating intensity of general activities, 15 activities assigned to 3 intensity classes: Light intensity activities (< 3.0 METs) lie, sit, stand, drive car, iron, fold laundry, clean house, watch TV, computer work Moderate intensity activities ( METs) walk, cycle, descend stairs, vacuum clean, Nordic walk Vigorous intensity activities (> 6.0 METs) run, ascend stairs, rope jump, play soccer This exceeds the potential of existing classifiers Aminian 2004

29 ISWC ConfAdaBoost.M1 III the more confident the weak learner is in an instance's correct classification / misclassification the more that instance's weight is reduced / increased weak learner returns the confidence of the classification estimation the more confident the weak learner is in a new instance's prediction, the more it counts in the final combined classifier

30 32 A new approach Test on the UCI database Diversity in how subjects perform physical activities personalization

31 21/11/ Personalization Basic idea: Everyone is different! Record your own data for a given activity over only one minute Summary Other Lying Sitting Standing Record about 6 out of 15 activities Alerts Upload Walking Running Cycling Nordic walking Re-arrangement of the classifiers (on the phone) Improve your recognition score

32 Cloud Service (e.g. SOS) Visualize data with GeoVisualizer Smart phone to track internal and external data / information

33 12/6/ TrackMe Overview Tracks, logs, and uploads (offline and online) of data from sensors (internal and external) Acts as a data logger, records data, and calculates (low and high-level) information even when not connected to the internet / cloud Retrieves data via Bluetooth from external sensors Retrieves data from available internal sensors, e.g. accelerometer, GPS, Gyro, Light, Audio Computing the following low and high-level information Push data manually to the cloud from within the TrackMe app or automatically upload for real-time monitoring

34 GeoVisualizer Overview (cont.) Developed in the European project SUDPLAN ( Different end-user requirements (e.g. data format, visualization, color mapping,...) Easy-to-use toolkit Licensed as open-source application under the LGPL version 3.0 Latest stable release Air-Quality Results Iso-Surface Visualization Visualization of Bio-Feedback 40

35 Continuous real-time analysis and visualisation SUDPLAN Sustainable Urban Development Planner for Climate Change Adaptation

36 Outlook Current focus of research Video / image analysis: large scale application Actvitiy monitoring: long term autonomy Out interest in this workshop Learn from experience in other field (bio-informatics, ) Transfer our approaches to other imaging or data modalities (bio-medicine,..)

37 Copyright 2013 Augmented Vision - DFKI Thank you for your attention! DFKI GmbH Department Augmented Vision T rippstadterst r. 122 D K aiserslautern Didier Stricker [email protected]