Counting animals from above: technical advances to monitor populations from flying platforms

Transcription

1 Counting animals from above: technical advances to monitor populations from flying platforms Peter Reinartz, Martin Wegmann Remote Sensing Technology Institute (IMF) Earth Observation Center (EOC) German Aerospace Center (DLR)

2 Automatic detection and counting already done in other disciplines Aim: Detecting and counting animals automatically 2

3 The primary goal: Crowd detection in mass events: Remote Sensing Technology Institute Detection of crowds and measuring the density of people may give information for the prevention of accidents. Understanding behavioral dynamics of large people groups can help to estimate future status of public areas. 3

4 Our technical equipment: Airborne camera systems Our primary aim and approach: Crowd detection and monitoring Transfering it to conservation: Animal detection: first results 4

5 Self-build system for large area coverage: Off-the-shelf: 3K Camera System (3 Canon EOS 1Ds Mark III, 21MPix) Max. framerate: 5Hz Realtime INS / D-GPS onboard Oblique angle max 55 Traffic modus: 3x bursts with pause Mapping modus Continuous acquisition Direct georeferencing Position accuracy < 1 meter flight height Coverage 3000 m 8.5 km m 2.8 km 0.13 m (1:60.000) (1:20.000) Coverage 5

6 Microwave link (up and downlink) for real-time monitoring Portable system Transmittance up to 100km Up to 40 Mbit/s 6

7 Example with two flight strips from 1000m a. g. Orthomosaik with pixel size 13cm 7

8 After the processing: Orthomosaik including radiometric adaptation 8

9 > Lecture > Author Document > Date Mass events Chart 10 Remote Sensing Technology Institute Visit of the Pope European Champions League Finale 19. May

10 Oktoberfest München, 0.5 sec 11

11 Car Detection and Traffic Monitoring 12

12 Also possible: 3D-Model from 3K-Data 13

13 Camera motion compensation and moving object detection on images from Octocopter Falcon 8 Folie 14 14

14 Local feature extraction change of color components in the place where a person (animal) exists. Extract local features from invariant chroma bands of the image. Input image colorspace conversion: RGB Lab Intensity Chroma Bands (Used to extract local features) For local feature extraction, we use features from accelerated segment test (FAST) which is mainly developed for corner detection using machine learning approaches, but also gives very high responses on small regions which are significantly different from surrounding pixels. Extracted local features from both a and b chroma bands E. Rosten, R. Porter, T. Drummond, Faster and better: A machine learning approach to corner detection, IEEE Transactions on Pattern Analysis and Machine Learning, Vol. 32 (1), pp , Nov Extracted Stadium 1 local test image features 15

15 Adaptive Kernel Density Estimation for Crowd Detection No pre-information about land cover Probabilistic framework to detect crowds. Extracted local features behave as observations of the probability density function (pdf) of the crowd to be estimated. Probability density function is computed as; Bandwidth of Gaussian kernel (smoothing parameter) Normalizing constant Euclidean distance to three nearest neighbor feature locations. Mean of three distances (l) is used to define variance of Gaussian kernel (σ 2 ). As tolerance, we select σ 2 = 5l Detected Extracted p(x,y) dense local matrix crowd features (color regions coded) 16

16 Adaptive Kernel Density Estimation for Crowd Detection: The estimated pdf gives information about crowded regions, and also helps to extract quantitative measures. Probability density function is computed as; Bandwidth of Gaussian kernel (smoothing parameter) Normalizing constant Otsu s automatic thresholding method is applied to pdf to detect regions having high probability values. In this case regions smaller than 1000 pixels are eliminated, since they cannot indicate large human crowds. N. Otsu, A threshold selection method from gray-level histograms, IEEE Transactions on System, Man, and Cybernetics, Vol. 9 (1), pp , Detected dense crowd regions 17

17 Experimental Results Dataset includes two different open-air concert images, an Oktoberfest image, and a stadium entrance dataset which includes 43 multitemporal images. Images are obtained from 3K airborne camera system. PERFORMANCE TABLE Comparison of groundtruth and automatically detected results. N = Number of People d = density (number of people per square meter) REGION 1 REGION 2 REGION 3 REGION 4 Labels of test regions in Stadium1 test image (used for performance evaluation) N N gth d d gth Mean of results counted by three human observers 19

18 Experimental Results: Oktoberfest Oktoberfest complete test image *Sirmacek, Beril und Reinartz, Peter (2011). Automatic crowd density and motion analysis in airborne image sequences based on a probabilistic framework. ICCV Nov. 11, ARTEMIS-Workshop, Barcelona, Spain 20

19 21

20 Tracking of single persons 22 22

21 Local density estimation, Rock Concert, Rock am Ring 23

22 Application of these (and more) algorithms in animal counting: 24

23 Animal detection and counting, first examples Remote Sensing Technology Institute Sample Images Extracted Local Features Automatic Detection Results Shadows 26

24 Challenges: dense agglomeration and shadows Remote Sensing Technology Institute Sample Images Extracted Local Features Automatic Detection Results Shadows 27

25 Challenges: partial detection and large crowds Remote Sensing Technology Institute Sample Images Extracted Local Features Automatic Detection Results 29

26 Animal detection and counting, first examples Remote Sensing Technology Institute Aerial survey in Kazahstan of Saiga Antelope (Steffen Zuther) 30

27 Animal detection and counting, first examples Remote Sensing Technology Institute 31

28 Animal detection and counting, first examples Remote Sensing Technology Institute count:33 missing: 2 32

29 Aerial wildebeest survey in Serengeti (Grant Hopcraft) 33

30 Animal detection and counting, first examples Remote Sensing Technology Institute count:35 missing: 1 34

31 Challenges: false detection background signal Remote Sensing Technology Institute count: 118 false detection:

32 Animal detection and counting in video, first examples Flying Foxes Remote Sensing Technology Institute 36

33 Thermal detection with UAVs, fawn detection in Germany 38

34 Questions and Challenges Remote Sensing Technology Institute For which purpose an airborne system could be used in conservation? With 3K-System: ~3000 km² within 5 hours Which are the basic conditions given in the monitoring area? Size of area Vegetation cover, shadowing effects Aircraft situation Accuracy of counting what is needed by conservation? Tracking of animals any application for that? Other monitoring possibilities: Low level cameras (no IMU, GPS) Thermal cameras UAVs Satellites 40

35 41