Advanced Methods for Pedestrian and Bicyclist Sensing

Transcription

1 Advanced Methods for Pedestrian and Bicyclist Sensing Yinhai Wang PacTrans STAR Lab University of Washington Tel: For Exchange with University of Nevada Reno Sept. 25, 2015

2 PacTrans Introduction Pacific Northwest Transportation Consortium (PacTrans) $3.5 million/year funds from USDOT Oregon State University University of Alaska Fairbanks University of Idaho University of Washington (lead) Washington State University PacTrans theme: Safe and sustainable solutions for the diverse transportation needs of the Pacific Northwest. 1

3 STAR Lab Smart Transportation Applications and Research Laboratory (STAR Lab): More Smart Transportation Applications and Research Website: Tel: STAR (7827) 2

4 STAR Lab Research Team 3

5 STAR Lab Activities 4

6 Age of Big Data Image sources: 5

7 Opportunities with Big Data Rich data for big decisions Baidu s 3.6 billion person trip prediction for China s Spring Festival passenger travel Image source: 6

8 Opportunities with Big Data Predicted big values $16.1 Billion Big Data Market: 2014 Predictions From IDC Marked as the beginning of a major transformation Those who can find the best story from the rich data available today will win. Those do not act on big data risk loosing ground to their competitors. McKinsey Global Institute concluded a $300 billion potential benefit for US Healthcare from using big data 7

9 Challenges with Big Data High volume In 2000, 800,000 petabytes (PB) of data stored in the world In 2012, 2.7 zettabyte (ZB, each ZB is 2 20 PB) By 2020, we expect this number to reach 35 ZB High variety Both structured and non-structured data: text, sensor data, audio, video, transactions and more Some data objects are not easily or efficiently stored in a relational database for queries, analysis, or other management tasks. High Velocity Data need to be stored and analyzed at the speed of data flow Privacy Other issues 8

10 Extra Challenges to Transportation Professionals Transportation theory Mathematical equation driven Based on small and possibly biased samples Location specific Lack of field validation and application guidance Transportation professionals Mostly trained as civil engineers Weak in IT and database knowledge Typically make empirical decisions without sufficient data support Static and isolated system view Transportation infrastructure Instrumented to some extent, but no general guidance on using the sensor data Data are not efficiently shared 9

11 Research Needs Proposed actions to address the gaps: Actively pull what we need from the existing data resources Build our own stream of big data Design a standard mechanism for connecting transportation related datasets Develop e-science transportation methods to take advantage of the spatial and temporal datasets to support transportation analysis and decision making Build big data analytics tools to facilitate usage of big data Develop new courses in transportation curriculum to make our students ready for the big data era 10

12 Research Needs What is e-science? Computationally intensive science that is carried out in highly distributed network environments Application of computer technology to the undertaking of modern scientific investigation E-science of transportation: Computationally intensive science for scientific investigations in transportation issues using immense data sets 11

13 Sources of Data Traffic Sensors Transportation Big Data! 12

14 Data Challenges Missing data, particularly pedestrian and bicycle data! WSDOT Pedestrian/Bicycle Count Report 13 Image source:

15 Data Challenges Image sources: 14

16 Background of Concept B1 B2 B3 B4 B7 B3 B6 Location x, t1 Location y, t3 B5 B2 B4 B5 B6 B8 15 Location x, t2 Location y, t4

17 Data Extraction from Mobile Networks 60% of the world population has a cell phone 91% of Americans 257 million data-capable devices in US (50 mil smartphones) 90% of Chinese in countryside areas Devices as data sources Count devices, not cars Devices as sensors Almost no maintenance Automatically scaling Ability to relay data back KNOWLEDGE + 16

18 Data Extraction from Mobile Networks Data collection paradigm scanning for devices Single Sensor r PERVASIVENESS + Population characteristics Multiple Sensors Population characteristics Travel characteristics Mobile Sensors Population characteristics Travel characteristics Interactions r r r r r 17

19 Data Extraction from Mobile Networks A mobile app based approach independent from cell carriers Dynamic network of monitors Report your immediate neighbors and your position Meet other monitors and report on them as well Increased pervasiveness Easy distribution Increased coverage Increased knowledge Which types of devices interact most? Where? 18

20 Opportunities Enabled by Mobile Network Data Social behavior Social pattern Transportation Route selection Pedestrian movement OD estimation Safety analysis Etc. 19

21 STAR Lab Research Examples Smartphone App (Mobile Monitor) Scans the Bluetooth spectrum Writes down GPS coordinates and MACs seen *phones used in testing courtesy of Dr. Borning 20

22 Opportunistic Sensing Issues Opportunistic Sensing Data Sparseness Population Uncertainty Temporal Uncertainty Spatial Uncertainty Volume Error Population Bias Travel Time Error Dwell Time Error Trajectory Error

23 Population Uncertainty: Solutions Mitigating Sparseness Filtering by device type Filtering by movement speed Discriminating via sensor placement Mitigating privacy risk MAC address deletion (replacement by order of arrival identifier) Recording of only the first 5 hex digits (k = 256)

24 Temporal Uncertainty: Solutions Mitigating Sparseness Increasing sample size with larger detection zones Outlier filtering (moving median) Historical data-based estimation during low volume intervals Mitigating privacy risk Aggregating records into bins (k-anonymity dependent on sample size)

25 Spatial Uncertainty What is the actual route taken?

26 Mitigating Spatial Uncertainty GIS-routable network (distance cost) Established popular routes using certain trajectories (popularity cost) Output most likely route for uncertain trajectories Heuristic-based routing algorithm Ranking the set using trajectory data More popular routes get lower costs Cost = f(distance, popularity)

27 PacTrans STAR Lab Research on Big Data and E-Science Spatial Uncertainty: Solutions

28 Verification by Simulation Can we representatively recreate true trajectories? How many observers do we need to recreate the true trajectories? How does the environment affect the usefulness of the approach? How do the protocol characteristics affect the approach? 27

29 Verification by Simulation 28

30 % Objects Observers Preliminary Simulation Results %Observers vs. %Detected y = e x Grid Street Expon. (Grid) Expon. (Street) % Objects Detected y = e 0.034x 29

31 Another Important Data Source 30

32 Current Methods for Pedestrians/Cyclists Sensing Manual count Pedestrian push buttons Infrared Inductance loops Pressure and acoustic mats Video image processing

33 Current State of Video Image Processing Human detection in video imagery is a long-standing computer vision challenge A great deal of current work is focused on featurebased detection Train machine learning classifiers for identifying local image features corresponding to humans or body parts Example: Histogram of Oriented Gradients (HOG) A number of algorithms have been developed for resolving occlusion, still a persistent challenge

34 Pedestrian Detection Under Occlusion Easy detection - Face and limbs clearly visible - Distinct from background Difficult detection - Obscured by environment or other people - Noisy environment

35 RGB-D Based Human Detection Background subtraction to extract pedestrian contours from RGB image simple and well studied Morphological processing to reduce noise and clutter in binary image Fuse RGB and Depth images Search for depth discontinuities within pedestrian blobs to resolve occlusion Pattern matching for people tracking Update count when people cross a depth threshold Implemented in C# with EMGU OvenCV 2.4 and Microsoft Kinect SDK 1.6

36 Microsoft Kinect Solution? INFRARED DEPTH SENSOR ARRAY RGB (COLOR) CAMERA MOTORIZED TILTING BASE ALSO: ACCELEROMETER AND MICROPHONE ARRAY

37 Field Experiments Note depth difference in occlusion instance

38 Field Experiments

39 Testing Results Scenario Test length Manual counts Under counting Over counting Accuracy (%) STAR Lab, cluttered indoor scene 5 min Staircase landing direct sunlight 5 min Open courtyard cloudy 5 min

40 What Have We Accomplished? Developed a RGB-D pedestrian detector using a low-cost consumer grade sensor Address the occlusion issue by fusing depth and color images Demonstrated good counting accuracy in both indoor and outdoor environments Demonstrated the utility of the Kinect outside of the manufacturer specified distance range

41 Concluding Remarks Sensor development and testing Bluetooth is a promising data source Sensors & app built, tested and tested again! RGB-D sensor is built to collect ped/bicyclist data Novel data collection paradigm Mobile sensors provide comparable data at lower cost DriveNET capable of analyzing and visualizing mobile data Kinect can be a cost-effective way of collecting pedestrian data in congested scenes MAC data analysis framework Can mitigate data sparseness and privacy risk Specific strategies developed and implemented 40

42 Thanks for your attention! Acknowledgment This project is partly funded by Washington State Department of Transportation (WSDOT) and PacTrans. We appreciate their funding support! 41