Real-Time Airport Security Checkpoint Surveillance Using a Camera Network

Transcription

1 Real-Time Airport Security Checkpoint Surveillance Using a Camera Network Richard J. Radke Department of Electrical, Computer and Systems Engineering Rensselaer Polytechnic Institute This material is based upon work supported by the U.S. Department of Homeland Security under Award Number 2008-ST-061- ED0001. The views and conclusions contained in this document are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied of the U.S. Department of Homeland Security.

2 Overview We constructed a full-scale simulation of an airport security screening checkpoint. The simulation environment was observed by a network of 19 cameras. We designed algorithms for the accurate real-time tracking of all passengers and bags, automatically maintaining which bags belonged to each passenger. The overall goal is to automatically verify expected behavior and detect abnormal behavior.

3 Previous Work

4 Simulation Environment

5 Simulation Environment

6 Simulation Environment

7 Camera Network

8 Camera Network Calibration LEDs OFF LEDs ON

9 Camera Network Calibration a. Calibrate all the PTZ cameras. b. Calibrate fixed cameras in groups with respect to different PTZ cameras. c. Unify all the cameras to the world coordinate system. RWCi RPCiRP RWP T R R T T WCi PCi P WP PCi

10 Camera Network Calibration

11 Undistortion and Color Correction Undistortion Color Correction - Histogram Specification Original Target Result

12 Stitching Images A D C B

13 Foreground Detection

14 Passenger Tracking Merge and Split Example #1 Connected Component Centroid Bounding Box Ellipse has the same 2 nd moments as the region Main Axis of the Ellipse Example #2

15 Baggage Tracking Dark Bag Detection Input Foreground Gray Scale Output Histogram Equalization Mean on X along Y Binarization

16 Bag Splitting Baggage Tracking n = 1,2,3,4,5 > 2X size of regular bin

17 Baggage Association

18 Baggage Association Get new frame Previous frame Yes No Exiting Object? Delete and change label to Pick Up Subtraction Update Tracker For each blob Match existing labels? Yes Update the positions No No END? Yes For each label Updated? Yes END? No New Object? Yes Assign new label Record Parameters No NO NO Small Delete Blob Detect Change Changed? YES For each Blob Size of Blob Normal Match a Label? YES Update Label END? YES Object Exiting? YES Remove from queue Pending Decision NO NO Big >2sec? Split Blob New Object? YES Bag Drop Area? NO NO Match label in Pending Decision? YES Insert matched label to queue YES NO YES NO Enter Pause State Assign New Label Delete Blob Bag-Drop Area Conveyer Belt Area

19 Baggage Association State Machine for Bag Labels New Exiting to the Staff area On Belt Entering from Pick-up Area Exiting to the Pick-Up area Pending Decision Exiting Pick-up Area & match Passenger Normal Taken away by staff Entering From Staff area Exiting Pick-up Area & Not matched Passenger Wrong Bag Alert!

20

21 Quantitative Results 4 runs of simulation, 25 minutes total Ground Truth Detected False Alarm Passengers Bags Normal Wrong Observed issues: ghosting from warping onto ground plane re-arrangement, re-use, and stacking of bins full occlusion of all bags at some point rearrangement due to rescanning

22 Current Work PTZ cameras Active targeting Families Even denser crowds Auto-calibration Statistics and flow

23 Future Directions Dense crowd motion analysis Anomaly detection (e.g., exit lane counterflow) Tag-and-track Sparse camera networks Integration with chem/rad/bio/range sensors More systemlevel testbeds

24 Conclusions We designed a full-scale airport security checkpoint surveillance system. We investigated the setup and calibration of a large camera network. We developed algorithms for real-time tracking and association of passengers and bags. The system is robust to crowded scenes and complex interactions. but what does John think?

25 Thanks Ziyan Wu Eric Ameres Keri Eustis DHS and ALERT John Pearson, Siemens ~rjradke/research