automatic road sign detection from survey video

Transcription

1 automatic road sign detection from survey video by paul stapleton gps4.us.com 10 ACSM BULLETIN february 2012

2 tech feature In the fields of road asset management and mapping for navigation, clients increasingly expect asset inventories and maps that are accurate and up-to-date. Keeping road sign databases current is a labour intensive, costly, and time-consuming exercise for government agencies and map producers. While video has become a commonly used tool for asset surveys in recent years, extraction of assets from that video continues to be a substantially manual process. Advances in the field of computer vision have been shown to hold promise in delivering automated object recognition tools that offer improved accuracy and reduced costs. However, genuinely automatic solutions for roadside asset inventory assembly from video have been scarce. of producing data outputs in a range of formats to integrate with existing map and sign inventory assembly operations. Before its introduction to North America, the Automapic technology was used to provide commercially viable analysis of more than 300,000 miles of city and rural roads to clients in the road and railway asset management and mapping markets. Daniel Lewis, Automapic s Vice President U.S. Operations, says this early commercial phase has helped the Automapic team to improve the technology s robustness and quality of results. The system is now part of various collection systems including Topcon, Earthmine stereo, and Ladybug variants. We have found that we can detect and geo-locate signs from virtually any video source synchronized with a GPS track he added. Video surveys of streetscapes or urban scenes are a common method for capturing data on the location of signs or other objects of business interest. Technology recently introduced to the US market may reverse this situation. An automated road sign detection and geo-location system, known as Automapic, has emerged from research and technology development undertaken by NICTA, Australia s leading ICT research institute. GeoNav Group International, Inc. of Great Falls, Montana, has recently adopted the Automapic technology and is now offering comprehensive road sign inventories to their mobile survey clients. The technology is highly scalable, capable The sign detection technology has been developed by NICTA s Computer Vision research group based in Canberra, Australia. Aware of the rapidly growing volume of video being collected by surveying and mapping firms, the group saw an opportunity to apply their expertise in object recognition, motion estimation, and machine learning to solving the real-world problem of how to efficiently extract inventories of roadside assets from video. They developed new algorithms for rapid object detection in video and february 2012 ACSM BULLETIN 11

3 MPEG motion estimation diagrm ( Example of object recognition (crim.ca) for triangulation of objects for mapping to realworld coordinates. Readers familiar with the field of machine learning will be aware of the use of a cascade of boosted weak classifiers, commonly applied to solve object recognition problems. The NICTA team utilizes and improves upon these methods and has filed patents covering novel object detection methods that are extremely fast to compute and highly discriminative. Another key element of the technology is the classifier compilation system which builds classifiers for each distinct sign type from a series of sample images taken from a training data set. The system can be trained to recognize most sign types found on roads worldwide. Accurate sign detection solves only part of the problem. Reliable sign position estimation is also vital for assembly of a high-quality sign inventory. The Automapic system employs a technique known as visual odometry, or camera tracking over large regions, to reliably estimate the camera path. Using the vehicle s GPS track, and other positional data such as from an inertial navigation system if available, the system will detect a sign over several frames, estimate the camera motion, then triangulate the position of the sign to establish its latitude and longitude. Visual camera tracking over large regions also makes it possible to better survey regions where there are GPS drop-outs, which is common, for example, in urban canyons. While largely dependent on the quality of the input data, the accuracy of sign post positioning can be within +/- 1 meter. Image sequence showing features tracked to measure changes in the position of the MER Opportunity rover on Mars during driving in soft, sloped soil (www-robotics.jpl.nasa.gov) Video recordings in downtown San Francisco, Cal. The colored path represents ground truth; GPS recordings are in black (Imt_ei_tum_de) 12 ACSM BULLETIN february 2012

4 tech feature Recent benchmarking in the research domain has shown sign detection methods based on visual localization to be best in class, maintaining very high detection rates, usually over 99%. WhereIs supplies up-to-date information about the way the road curves and slopes and its lane markings, and thus helps deliver lane guidance and traffic information to drivers via mobile GPS. allowed us to evolve our process for capturing signs and coding the database, further improving the completeness, accuracy and quality of our digital mapping products. NICTA Automapic also works in partnership with leading mobile and spatial technologies company Geomatic Technologies (GT) to deliver road asset inventories. The City of Greater Geelong, Victoria s largest provincial centre with 2500 km of urban and rural roads, recently needed to assemble an inventory of road signs to meet legal requirements. They contracted GT to undertake a survey using their vehicle-mounted Earthmine system which collects 3D panoramic imagery. To ensure a high quality road sign inventory, GT engaged the Automapic team to analyse the stereographic street level imagery to identify and locate more than 100 different types of road signs and roadside hazard markers, including regulatory, caution and warning signs. The project delivered GT s rapid field capture program with earthmine 3D imagery in City of Greater Geelong, Victoria Australia (geomatics.com.au). A couple of brief case studies on the use of the technology to date may help illustrate its potential. Sensis, Australia s leading provider of navigable maps, and a long term client of the Automapic team, uses the automatic video analysis technology to help build and maintain its national mapping database. General Manager of the Sensis mapping business, Peter Barclay said, The Sensis Whereis digital map data is behind most of Australia s online, in-car, portable and mobile satellite navigation products. We re very proud of our database and we re constantly on the lookout for opportunities that will assist us in continuing to maintain the database to the highest standard. Our agreement with NICTA has to GT an inventory of more than 20,000 individual road signs classified in accordance with Australian standards. Now being applied in the U.S. market, GeoNav Group International, Inc. President Guner Gardenhire, is confident that the technology will deliver similar benefits. He says Automapic will help them respond to a growing need for reliable road sign surveys to address maintenance, safety and regulation issues. In particular there is an increasing demand for up-to-date sign inventories to address the Federal Highway Administration s new traffic sign retroreflectivity requirements aimed at reducing the disproportionate number of traffic fatalities that occur on American roads at night. february 2012 ACSM BULLETIN 13

5 GeoNav Group deploy vehicle mounted Topcon IP-S2 systems on a fleet of trucks operating nationwide. While much of their business until recently has been servicing the rural utilities sector, they are addressing an increasing requirement for quality road asset inventories in the local and state government sectors. Their sign inventory service offers a current library of more than 70 detectable U.S. sign types. The automatically generated sign inventories, configured to suit individual client needs, typically include, as key attributes, the type of sign named according to the appropriate MUTCD standard, as well as latitude, longitude, and orientation of the sign face. NICTA is also working with GeoNav Group to implement new object-recognition technologies that utilise 3D point clouds generated by Topcon laser scanners to improve the productivity of their power pole inventory assembly operations. Computer vision technologies are already helping surveying and mapping companies to respond to the growing need for efficient creation of roadside the opportunity to develop techniques that draw on the particular strengths of each data source, both individually and in combination. Forthcoming developments include LIDAR point clouds for improved Road signs geo-positioned by the Sign Geo-positioning Service. [O Connor, Canberra, ACT, Australia] asset inventories to address maintenance, safety and regulation issues. In the future we can expect to see further automation of object detection and classification in surveying operations. Systems that capture both video and LiDAR provide positioning of road signs and other assets, and the use of LiDAR to estimate sign reflectivity and recognition and classification of a wide range of objects from 3D point clouds. The NICTA team, along with others in the research community, are working on such challenges right now. 14 ACSM BULLETIN february 2012