Qu, Parker, Cheng, Ran, Noyce 1. Large Scale Intelligent Transportation System Traffic Detector Data Archiving
|
|
|
- Brittany Barnett
- 10 years ago
- Views:
Transcription
1 Qu, Parker, Cheng, Ran, Noyce Large Scale Intelligent Transportation System Traffic Detector Data Archiving Tao Qu, Graduate Research Assistant, Department of Civil and Environment Engineering, University of Wisconsin-Madison Engineering Drive, Room, Madison, WI, 0 Phone number: (0)0-, [email protected] Steven T. Parker, Ph.D, IT Program Manager, Department of Civil and Environmental Engineering, University of Wisconsin-Madison, 0 Engineering Hall, Engineering Drive, Madison, WI, USA 0, Phone: (0) -, Fax: (0) -, [email protected] Yang Cheng, PhD, Research Associate, Traffic Operations and Safety (TOPS) Laboratory Department of Civil and Environment Engineering University of Wisconsin-Madison Engineering Hall, Engineering Drive, Madison, WI, USA 0, Phone: (0) - [email protected] Dr. Bin Ran, Ph.D., Professor Department of Civil & Environmental Engineering, University of Wisconsin-Madison, Engineering Drive, Madison, WI 0, USA Phone: Fax: [email protected] and School of Transportation, Southeast University No. Si Pai Lou, Nanjing 00, China David A. Noyce, Ph,D, P.E., Professor Director, TOPS Lab Department of Civil & Environmental Engineering, University of Wisconsin-Madison, Madison, WI 0, USA Phone: -0-- Fax: [email protected] Submission date: //0 Word count:, + Figure + Tables =, words TRB 0 Annual Meeting
2 Qu, Parker, Cheng, Ran, Noyce 0 ABSTRACT Archived traffic data can be used in transportation planning, administration, and research by various entities and agencies. During the past two decades, considerable effort has been dedicated to developing and implementing large-scale traffic data archives. The Wisconsin Traffic Operation and Safety (TOPS) Laboratory at the University of Wisconsin Madison maintains a statewide traffic detector data archiving and retrieving system, which is developed to enable centralized management of statewide ITS detector and configuration data, optimizing the utilization of massive data on a systematic level, and improving the interactivity and accessibility for integration with other transportation data sources such as lane closure data or incident data. This data archive is currently being enhanced to incorporate higher resolution traffic data by migrating from -minute to -minute and even 0-second sampling intervals. At the same time, there is a desire to generate aggregated datasets such as hourly, monthly, and annual average values from the raw data. As the traffic data requirements continue to grow, the management of the traffic data archive becomes a complex big data problem. This paper describes a proposed redesign of the TOPS Lab traffic detector archived data management system to improve storage, performance, access, and integration capabilities. Particular detail is given to the data archiving process, including data validation, and support for spatial attributes and GIS data integration. TRB 0 Annual Meeting
3 Qu, Parker, Cheng, Ran, Noyce INTRODUCTION Traffic data archiving for Intelligent Transportation Systems (ITS) refers to the systematic retention and reuse of operational ITS data, and is required for many application. In the mid-0s, real-time data from traffic operations was archived and used for purposes beyond traffic control strategies. Archiving these otherwise discarded data provides a rich source of information for evaluating traffic flow characteristics and transportation system performance on a continuing basis (). To encourage the retention and reuse of ITS-generated data, the archived data user service (ADUS) element of the National ITS Architecture requires that data from ITS systems be collected and archived for historical, secondary and non-real-time uses and that these data be made readily available to users (). It also provides a general framework for collecting, processing, retaining, and distributing these data. As a result, during the past two decades, many data archiving activities have been conducted by different transportation agencies and considerable effort has been dedicated to developing and implementing large-scale traffic data archives (,, ). As a supplement or replacement of conventional data sources, the cost-effectiveness of data collection infrastructure is maximized, sampling bias is minimized due to the continuous collection of data, and a better understanding of variability in system performance can be accomplished (). Once these data are archived, the vast amount of data can be used in transportation planning, administration, and research by various entities and agencies including metropolitan planning organizations (MPOs), state transportation planners, traffic management operators, transit operators, and transportation researchers (). The Wisconsin Department of Transportation (WisDOT) has invested heavily over the past two decades in ITS infrastructure. This includes the emergence of a Statewide Traffic Operations Center (STOC) in Milwaukee and the deployment of Advanced Transportation Management Systems (ATMS) software at the STOC and other regional offices. The STOC gathers data from approximately,00 traffic detectors that compose the ATMS. A variety of point detection technologies are implemented on the state freeway system including: inductive loop detectors, radar detection, microwave detection, and more recently Bluetooth detection. Generally, these detection systems provide traffic flow data for freeway surveillance and real-time freeway management. Traffic detector data from the STOC ATMS is archived on a continual basis in the WisTransPortal system, located at the Wisconsin Traffic Operations and Safety (TOPS) Laboratory at the University of Wisconsin-Madison (,). The ATMS detector data is available for online query and retrieval through the WisTransPortal VSPOC system (). VSPOC serves as the primary source of archived Wisconsin ITS traffic detector data and is used by WisDOT, local governments, consulting firms and university researchers for a variety of traffic operations, planning and research purposes. Since deployment of the VSPOC system in 00, several significant challenges to the effective use of the WisTransPortal ATMS traffic detector data have been identified. The primary challenges relate to a lack of geo-spatial (GIS) location attributes, ongoing data quality issues, data frequency, and documentation. In addition, the sheer volume of data received from the STOC (over million records per day) presents ongoing data management considerations. This paper will provide a overview of current development efforts by TOPS Lab to upgrade the WisTransPortal ATMS detector data archive and VSPOC system to address these challenges. A specific focus on large scale data management and GIS integration will be discussed, along with implications for improved query capabilities and integration with other WisTransPortal transportation data. WISCONSIN ITS TRAFFIC DETECTOR DATA Wisconsin Traffic Data Sources As in other states, there are several sources of highway traffic detector data in Wisconsin: real-time traffic ITS detector data from the STOC control room ATMS, Traveler Information link speeds and travel times (derived from the ATMS detector data), continuous and short-term traffic counts from the WisDOT Bureau of State Highways (BSHP) planning area, and more recently Bluetooth sensor data. In addition, third party traffic data is available from a variety of sources. TRB 0 Annual Meeting
4 Qu, Parker, Cheng, Ran, Noyce The STOC ATMS monitors real-time traffic volume, speed, and occupancy data from approximately,00 freeway detectors in five transportation regions (Table ). These data are used for real-time management of the transportation system to monitor the traffic state and identify congested area and incidents. At 0-second intervals, each detector records vehicle counts, average speed, and occupancy (i.e., the percentage of the sample period when a vehicle was over the detector). A roadside controller is then polled at that same time interval by the central traffic management software. The ATMS aggregates 0-second data to -minute intervals for control room monitoring and traveler information purposes. As described, traffic detector data from the STOC ATMS in the TOPS Lab WisTransPortal system. The basic objective of the WisTransPortal is to develop capabilities for a statewide ITS data hub to support multiple applications in traffic operations and safety. Those capabilities include integration, management, analysis, and dissemination of real-time and historical ITS operations data through a centralized database and communications infrastructure (). The data archiving component of the WisTransPortal consists of a collection of automated services that connect to various WisDOT and other ITS data sources and prepare the data for archiving in a common relational database. TABLE Wisconsin Freeway Detector Distribution WisDOT Region Online Date Controllers Detectors SE Region SW Region 00 0 NC Region 00 NE Region 00 NW Region 0 The WisTransPortal traffic detector database is updated every hours with WisDOT ATMS detector data from the previous day. There are three major files are sent from the WisDOT ATMS to TOPS lab for archiving via the WisDOT "ITSNET" fiber network: ATMS Detector Configuration Inventory ATMS Controller Configuration Inventory ATMS One Minute Detector Volume, Speed, And Occupancy Data The design of the WisTransPortal detector database incorporated elements form the Traffic Management Data Dictionary (TMDD) standard (0) whenever possible. Hence, as the guiding principle, the WisTransPortal detector database is structured to support TMDD requirements. VSPOC Online Tool The V-SPOC web-interface includes a complete detector database query selection tool, data visualization and export capabilities, data quality reporting, and integration with other ITS data sets in the WisTransPortal database. In order to facilitate the use of this data, the current VSPOC application suite provides mechanisms to organize detector data into spatial (corridors, count locations and controllers) and temporal (time interval) groupings. Besides that, VSPOC was designed as a set of customized modules that take advantage of a common database backend and web-based application platform to provide an integrated system for transportation operations engineering, analysis, research, and reporting functions. Several specialized modules are available for various needs of users and suitable presentation of data including corridor analysis, monthly data retrieval, ramp metering retiming, etc. Figure presents an example of an archive volume plot for sample detector location on I- in Milwaukee County. TRB 0 Annual Meeting
5 Qu, Parker, Cheng, Ran, Noyce 0 Figure Example of VSPOC modular. VSPOC ETL Due to the complexity and distinct set of requirements, it is common to separate the implementation of real-time traffic management system software from the archived data management component. The process of populating the traffic data archive, or moving data from the operation center s database to the traffic data archive, is known as extraction, transformation, and loading (ETL). Incorporating effective ETL process and archiving algorithm for high resolution loop detector data is complex and requires significant attention in design. The current extraction and transformation process used to populate the VSPOC database system is presented in Figure. FIGURE WisTransPortal VSPOC detector data architecture. TRB 0 Annual Meeting
6 Qu, Parker, Cheng, Ran, Noyce 0 VSPOC Database Future Trends The STOC has recently starting providing ATMS detector data to the WisTransPortal in terms of - minute intervals. Converting from the previous -minute resolution to -minute resolution introduces a number of data management challenges. As of 0, the cumulative record count of traffic detector data in the WisTransPortal had already exceeded four billion. In addition, as the continuous development and investment on ITS devices, the scale of traffic monitoring operations will also grow. As presented in Figure, the cumulative record count in 0 is predicted to be nine billion, which means the amount of two years data are larger than that of past years. The linear trend of data size growth will be five times than before. Consequently, traffic data archiving is becoming a big data challenge which requires a mass data storage solution with high fault tolerance and throughput and sophisticated data management techniques. 0 FIGURE VSPOC database cumulative record count. ARCHIVING PROCESS The archiving process refers to steps taken for processing and archiving the traffic detector data and system configuration data. To solve the aforementioned big data challenge and support the VSPOC frontend web application, it is necessary to continue improving the effectiveness of the archiving system and data performance in terms of completeness, timeliness, accuracy, and interactivity. Redesigning the archiving process provides a unique opportunity to rethink how to better shape the VSPOC traffic data archive. Hence, the following three objectives are identified:. Automation: Increase and refine the automation capabilities of the archiving process. Data Validation: Implement front-end data validation steps and report exceptions that occur in archiving process. Aggregation: Transform the temporal and spatial grain of raw data to desired temporal and spatial grain of archived data The proposed archiving process is presented in flowchart in Figure. Included discussion is how these objectives are taken into account within the archiving process and how to address identified issues. TRB 0 Annual Meeting
7 Qu, Parker, Cheng, Ran, Noyce ITS fiber network TOPS Lab Software process Controllers STOC ATMS Table JAVA Program Errors in the raw data? No Load detector and controller configuration inventory and traffic data to corresponding loader table Impute missing and bad data based on historical average Direct insert into the target historical table respectively Yes Historical Traffic Data Table Historical Controller Configuration Table Historical Detector Configuration Table Densification Controller timeline Table Conflict in detectors? Yes No minute traffic data Eliminating conflict detector Detector timeline table min aggregate table Snapshot of current detectors min aggregate table 0 min aggregate table Daily archiving completed FIGURE Proposed VSPOC archiving process Location information for spatial correlation Snapshot of current detectors with location infomation Write results to log, and to operator TRB 0 Annual Meeting
8 Qu, Parker, Cheng, Ran, Noyce Automation A Java program is developed to implement the archiving process including retrieving source data from the STOC, importing the raw data into a temporary loader table, and processing the loader data to the target database. Once deployed, it can be used for centralized management. This process is completely automated it runs as a scheduled "cron" job on a separate virtual server assigned for data acquisition and writes a log of its activity. In the current VSPOC system, only the detector data import process is automated. The process for updating the detector and controller configuration data is a manual procedure. Moreover, the automated detector data archiving component does not incorporate a full loader process and is therefore limited in its post-processing and data validation capabilities, as described below. Data Validation Structural errors in the detector data files arise from system faults in the ATMS itself, differences in how detector data is represented in the ATMS compared to the WisTransPortal, and in the packaging and data transmission process. The Java archiving program handles structural errors in the input data to support high quality of data acquisition. In each step, specific constraints have been set for data validation. These constraints are used to automate the process of identifying and loading the qualifying data into the archive system. Rules are implemented to handle conflicts, where possible, in order to capture as much data as possible without compromising the archive. Moreover, all exceptions are caught and reported by . Checking and analyzing the reported errors enables developer to resolve them at first time, which aims at dynamic data maintenance and continuous improvement on VSPOC. Once in the archive system, data remains online and accessible. Aggregation Data aggregation can be used to reduce storage requirements on large datasets and to improve the performance of analytic processing over vast amounts of fine grain data. Whereas the temporal and spatial granularity at the traffic operations center is generally limited to point detections and travel time routes with respect to -minute time intervals, it is desirable for the VSPOC archive to supports a range of aggregation levels. The frontend of VSOPC suite is a web user interface for data query, data visualization, data exporting, quality reporting, and corridor analysis. By using physical tables to store traffic data at desired aggregation levels, the underlying database system can better support the various needs of VSPOC users such as providing monthly or annual averages of daily traffic patterns. In addition, data aggregation can smooth out irregular data values and eliminate noise that may report occasionally because of detector or communication failures. Pre-processing aggregate data and storing it physically in the database improves performance but increases data management requirements. A description of typical temporal and spatial granularity requirements in an operation center and a traffic archive with Wisconsin s practice is provided in Table. TABLE Grain Description Aggregation Level Operation Center s Database Traffic Data Archive Temporal Intervals between two successive polling operations of same detector by a controller. STOC: 0 seconds Travel times and link speeds for control room and traveler information: minutes Spatial Geographic area served by a detector. STOC: a transverse section of a lane. Intervals between two successive database records for the same detector location. VSPOC: -minute, -miunte, -minute, 0-minutes, daily, monthly. Transverse section of a lane, detector location, roadway segments such as link or corridor. VSPOC: lane, link, corridor. TRB 0 Annual Meeting
9 Qu, Parker, Cheng, Ran, Noyce 0 DATABASE DESIGN The new database design targets the VSPOC backend to address the identified issues in current archiving process and disadvantages of underlying database, as well as targets data product in terms of necessary improvement on data quality and data coverage. In particular, the following four objectives are the primary concerns during the redesign of database: ) Establish historical archives of system configuration data; ) Accomplish densification of traffic data of current online detectors; ) Synchronize internal ID and external ID for each detector (internal ID refers to an individual detector, external ID points to a physical detector location); and ) Enhance GIS attributes of detector. The proposed design makes sure the archived data are accurate, sufficient, and easily accessible through a high performance database system. The database environment for VSPOC is the WisTransPortal Oracle Database g Enterprise Edition, which can improve database performance and reliability. Table Design for Configuration Data The table design for configuration data primarily address the challenge of management on historical configuration data and synchronization and transformation between internal ID and external ID. Figure shows the data flow of system configuration data and relationship among each functional table. The archiving process leverages this design to automate data processing on system configuration data and to generate current snapshot of online detector. ATMS_DETECTOR_LOADER ATMS_DETECTOR_HISTORY ATMS_DETECTOR_TIMELINE ATMS_DETECTORS internalid orgid externalid internalid regiond (orgid) detectorid (externalid) internalid start_date end_date detectorid detectorid status controllerid createddate lastmodified ATMS_CONTROLLER_LOADER ATMS_CONTROLLER_HISTORY ATMS_CONTROLLER_TIMELINE ATMS_CONTROLLERS 0 0 orgid controllerid (orgid) controllerid controllerid start_date end_date controllerid status createddate lastmodified FIGURE Database design and data flow for detector (controller) configuration inventory. Staging Loader table Two intermediate staging databases (DETECTOR_LOADER and CONTROLLER_LOADER) serve as loaders to import detector system configuration data. The primary purpose of staging loader table is to accommodate the significant real-time process requirement when populating vast amount of data, providing maximum availability and stability during archiving process. As shown in the flowchart of archiving process, each night, the Java archiving program retrieves the system configuration data and then loads the data to specific loader table for the previous -hour period. As a buffer, the staging loader tables can process and validate the received data before it goes to archive, and prevent duplicate recorder in historical table. It ensures the further process operates on clean, correct and useful data. After inserting the data with into historical table, the loader tables will be emptied to recycle databases. TRB 0 Annual Meeting
10 Qu, Parker, Cheng, Ran, Noyce Historical table The VSPOC database system accumulates new configuration information but still needs to retain older information to preserve point in time snapshots of the ATMS system configuration. Two historical (DETECTOR_HISTORY and CONTROLLER_HISTORY) tables are then designed to keep historical information of system configuration to date. It provides an efficient way to update system configuration data on regular basis, and allows original data to be preserved to ensure complete, reliable integrity for the life of archived data. In addition, it prevents the potential risk from system crash or failure of Java archiving program. Backup and recovery runs faster based on historical table, disaster recovery therefore is less costly. Moreover, the historical tables enables further development upon raw data, facilitating diagnosis algorithm for detector health and data availability. After the data is inserted into historical table, qualified raw data is archived exactly as it is received in real time. The entire current day s data are then populated to the VSPOC database backbend s historical table in a nightly batch after the end of day. Timeline table The detector and controller timeline table (DETECTOR_TIMELINE and CONTROLLER_TIMELINE) present a timeline for each individual detector and controller. The need of timeline table stems from the detector management strategies. For each detector, it either can be identified by INTERNALID which is unique lifetime identifier for an individual detector, or the pair of REGIONID and EXTERNALID, a unique identifier for an individual detector within a given WisDOT transportation region (REGIONID) field detector data collection system. For example, if a detector is broken, a new detector with a different INTERNALID will be installed to replace the previous one. However, the EXTERNALID for this detector location remains the same. Figure concludes three types of detector ID conflicts. By grouping data by detector internal ID then ordering by, two new attributes are created, which are Start_Date and End_Date indicating online and offline date for a certain detector or controller (identified by INTELNALID or CONTROLLERID). By matching the online and offline date of each detector or controller to their corresponding EXTERNALID, traffic data can be precisely retrieved by locating the online detector for specific time and location. Table is an example of how timeline table solve conflict detector issue. Three types of conflict detector issue:. Overlap. Gap. Return The new detector becomes online before the current one become offline. There is an overlapping period. The new detector becomes online after the current one become offline. There is a gap between the offline date of the current detector and the online date of the new detector. The current detector become offline, after a period of time, it returns back and becomes online. Overlap Gap Return Detector Detector Detector Detector Detector Detector Detector Detector Detector 0 TIME TIME TIME FIGURE Illustration of three types of detector ID conflict TRB 0 Annual Meeting
11 Qu, Parker, Cheng, Ran, Noyce 0 0 TABLE Example of How Timeline Table Solves Conflicting Detectors INTID REGIONID EXTID FIRST REPORTED START_DATE END_DATE The design of the timeline table can also support the systematic retention of data generated from detectors over time. If any unexpected problems found in historical data, it can allow user easy to access the data and locate the problem. Other tables and views The database system also involves additional tables (views) in the database to maximum flexibility and availability. Two important tables are defined as: ATMS_DETECTORS: Current snapshot for detector configuration information by STOC. A status flag indicate whether the detector is current or historical. The value is either C for current or H for historic. VSPOC_DETETCORS: Additional detector configuration information including location information entered through VSPOC configuration interface. Table Design for Traffic Data The database design for traffic data ensures quality and availability of traffic detector data. As the same logic of configuration data processing, a loader table (ATMSREPORTS_LOADER) is used to filter bad record in the raw data, while ATMSREPORTS_MIN serves as historical table for traffic detector data. Figure shows the data flow of traffic detector data, procedures for handling conflicting detectors, and densification. ATMS_DETECTOR_TIMELINE internalid state_date end_date Resolve configuration conflict V_ATMSREPORTS_MIN detectorid sampleduration samplestarttime volume speed ocuupancy DETLOGS.csv Internalid volume speed ocuupancy ATMSREPORTS_LOADER internalid volume speed ocuupancy ATMSREPORTS_MIN internalid volume speed ocuupancy V_ATMSREPORTS_MIN detectorid interalalid volume speed ocuupancy V_ATMSREPORTS_MIN detectorid sampleduration samplestarttime volume speed ocuupancy Densification V_ATMSREPORTS_0MIN ATMS_TIMESTAMP FIGURE Database design and data flow for traffic detector data. detectorid sampleduration samplestarttime volume speed ocuupancy TRB 0 Annual Meeting
12 Qu, Parker, Cheng, Ran, Noyce Densification It is common for the detector logs to exhibit short-term and long-term break in data availability in terms of discontinued time series. In order to provide accurate reporting and analysis functions through VSPOC, it is desirable to work with a dataset that has a uniform sample space. To generate a complete set of records for all time periods, a pre-defined template table consisting of -minute s was created to join with the source detector data. To screen out offline detectors, the raw data join the DETECTOR_TIMELINE table first. Then by joining the table, a database view (ATMSREPORTS_MIN) is generated which provides a dense presentation of traffic data of current online detector by filling in NULL values at s where archived data is missing. All detector reports table must be uniform in its representation of time, by doing this, data densification is pushed down to the database level where it most efficiently solved. GIS Attributes As described, the WisTransPortal design incorporates two functions: ITS data archiving and ITS data dissemination. The current data archiving capabilities are primarily focused on integrating and correlating six existing WisDOT and regional data sources: traffic detector data, lane and ramp closure data, traffic incident data, historical crash reports, freeway traffic video, and road weather condition. Each subsystem is designed according to high level architecture to be a component of a highly-integrated, highlyinteractive traffic management database system. As mentioned, the VSPOC_DETECTOR table contains location information for detectors. Besides using a latitude and longitude pair to represent a detector location, it can be modeled by a single control point implemented by a system of fixed landmarks. A landmark is defined as a physical, identifiable point on a highway, such as an intersection, milepost, bridge, or a virtual point, such as a county boundary line over a highway. In VSPOC, landmark and corresponding offset, which comprise a direction and a distance, can be used to provide highly accurate location for detectors on Wisconsin State Trunk Network (STN), the WisDOT GIS-based linear referencing system for state and federal highway in Wisconsin. These processes enable the new VSPOC system to incorporate data from internal WisDOT data systems and applications with minimal user interaction. The integration of data is the key to next generation data archived and management system as a single data source correlated with other related data sources. In addition, TOPS lab is on progress to visualize traffic detectors location by mapping all the detectors on STN linear referencing system. Figure shows the mapped detector location on STN linear referencing system. TRB 0 Annual Meeting
13 Qu, Parker, Cheng, Ran, Noyce 0 0 FIGURE Mapped detector on stn linear referencing system. CONCLUSION This study describes experience and implementation on large-scale ITS traffic detector data archiving. The identified issues and potential solutions are universal and therefore other regions around the country can be benefit from this experience. Towards a next generation of traffic data archive, the VSPOC system will continue dedicating on data availability and accessibility for research and decision support. Future work of this study may include following directions. First, a comprehensive algorithm is needed to optimize system performance. Second, by identifying attributes of data quality that are relevant to ITS data archiving, data quality assessment and statistical techniques are desirable to improve data quality. Third, continuous improvements are needed on GIS network, data modeling, and high level system integration to leverage other data sources such as lane closure data or incident data to support decision making. ACKNOWLEDGMENT The WisTransPortal system and VSPOC traffic detector data archive were developed through sponsorship of the Wisconsin Department of Transportation Bureau of Traffic Operations (BTO). Ongoing data archiving and system enhancement activities with respect to the ITS traffic detector data are conducted by TOPS Lab in coordination with the BTO Statewide Traffic Operations Center. REFERENCES TRB 0 Annual Meeting
14 Qu, Parker, Cheng, Ran, Noyce 0 0. Turner, S. M. Guidelines for Developing ITS Data Archiving Systems. Report -. Texas Transportation Institute, Texas A&M University. System, 00.Archived Data User Service (ADUS): An Addendum to the ITS Program Plan. ADUS Program. U.S. Department of Transportation, Sept... Brian L. Smith and Simona Babiceanu. Investigation of Extraction, Transformation, and Loading Techniques for Traffic Data WarehousesFederal Highway Administration. In Transportation Research Record: Journal of the Transportation Research Board, No., TRB, National Research Council, Washington, D.C., 00, pp... Robert L. Bertini, Steve Hansen, Andrew Byrd, and Thareth Yin. Experience Implementing a User Service for Archived Intelligent Transportation Systems Data. In Transportation Research Record: Journal of the Transportation Research Board, No., Transportation Research Board of the National Academies, Washington, D.C., 00, pp. 0.. Dilruba Ozmen-Ertekin, Kaan Ozbay. Dynamic Data Maintenance for Quality Data, Quality Research. OECD. Presented at 0th Annual Meeting of the Transportation Research Board, Washington, D.C., 0.. ITS Data Archiving: Five-Year Program Description. ADUS Program. U.S. Department of Transportation, March Parker, Steven. and Tao, Yang.WisTransPortal: A Wisconsin Traffic Operations Data Hub. Applications of Advanced Technology in Transportation (00): pp. -.. Wisconsin DOT, TOPS Laboratory UW-Madison. V-SPOC Traffic Detector Database Tools. TOPS, Wisconsin Traffic Operations and Safety Laboratory The Wistransportal Project 0. Institute for Traffic Engineers (00). Traffic Management Data Dictionary (TMDD), Version.. TRB 0 Annual Meeting
Intergraph Roadway Information Management Solution. Title Title. Title Title. A White Paper
Intergraph Roadway Information Management Solution A White Paper Security, Government & Infrastructure, a division of Intergraph Title Title Title Title Table of Contents 1. Introduction... 1 2. Intergraph
Presented by: Dr. Senanu Ashiabor, Intermodal Logistics Consulting Inc. at 2015 North Carolina MPO Conference April 30, 2015
Presented by: Dr. Senanu Ashiabor, Intermodal Logistics Consulting Inc. at 2015 North Carolina MPO Conference April 30, 2015 What is Big Data? Big data is a broad term for large datasets that are so large
Metropolitan Intelligent Transportation Systems (ITS) Infrastructure 2010 Transportation Management Center
Metropolitan Intelligent Transportation Systems (ITS) Infrastructure 2010 Instructions This questionnaire is designed to obtain data measuring the level of Intelligent Transportation System (ITS) implemented
Integrated Data System Structure for Active Traffic Management - Planning and Operation
Integrated Data System Structure for Active Traffic Management - Planning and Operation NATMEC 2010 Dr. Xiao-Yun Lu, Research Engineer California PATH, U. C. Berkeley J. Palen, Caltrans DRI 1 Outlines
Use of a Web-Based GIS for Real-Time Traffic Information Fusion and Presentation over the Internet
Use of a Web-Based GIS for Real-Time Traffic Information Fusion and Presentation over the Internet SUMMARY Dimitris Kotzinos 1, Poulicos Prastacos 2 1 Department of Computer Science, University of Crete
A PHOTOGRAMMETRIC APPRAOCH FOR AUTOMATIC TRAFFIC ASSESSMENT USING CONVENTIONAL CCTV CAMERA
A PHOTOGRAMMETRIC APPRAOCH FOR AUTOMATIC TRAFFIC ASSESSMENT USING CONVENTIONAL CCTV CAMERA N. Zarrinpanjeh a, F. Dadrassjavan b, H. Fattahi c * a Islamic Azad University of Qazvin - [email protected]
Primavera Project Management System at WVDOT. Presented by Marshall Burgess, WVDOT Stephen Cole, Stephen Cole Consulting Jervetta Bruce, CDP, Inc.
: Integrating GIS With Primavera Project Management System at WVDOT Presented by Marshall Burgess, WVDOT Stephen Cole, Stephen Cole Consulting Jervetta Bruce, CDP, Inc. WVDOT Geographic Information System
Public Sector Solutions
Public Sector Solutions The New Jersey DOT Command Center uses INRIX real-time data and analytics to monitor congestion and deploy resources proactively to manage traffic flow and inform travelers. INRIX
How To Design A Crash Investigation Site
CHAPTER 7 CRASH INVESTIGATION SITES December, 2000 7. Crash Investigation Sites (CIS) 7.1. Introduction and Usage The idea of "crash investigation sites" or areas outside of the freeway mainline where
OM-13: Transportation Management and Operations
: Transportation and Operations 1-15 points Goal: Maximize the utility of the existing roadway network through use of technology and management of operations strategies. Sustainability Linkage Transportation
FINAL REPORT DEVELOPMENT OF CONGESTION PERFORMANCE MEASURES USING ITS INFORMATION. Sarah B. Medley Graduate Research Assistant
FINAL REPORT DEVELOPMENT OF CONGESTION PERFORMANCE MEASURES USING ITS INFORMATION Sarah B. Medley Graduate Research Assistant Michael J. Demetsky, Ph.D., P.E. Faculty Research Scientist and Professor of
BlueTOAD Bluetooth Travel-Time Monitoring
BlueTOAD Bluetooth Travel-Time Monitoring Presentation Outline What is BlueTOAD BlueTOAD Applications over 600 units deployed BlueTOAD Website: Data Outliers Extracting MAC addresses Real-time application
Traffic Incident Management Enhancement (TIME) Blueprint Version 2.0 Executive Summary
Blueprint Version 2.0 Executive Summary Strategic Background The Southeastern Wisconsin region, which encompasses the following eight counties: Fond du Lac, Kenosha, Milwaukee, Ozaukee, Racine, Walworth,
Y R T S TransporTaTion U D IN solutions
I N D U S T R Y Transportation Solutions Keeping people and products moving safely and efficiently In a world of constant change, keeping people and products moving safely and efficiently is a complex,
Eastern RTMC. www.dot.state.pa.us
Eastern RTMC Regional Traffic Management Centers Agenda District 6 0 TIMS Program Overview Freeway and Arterial ITS Deployments Ongoing/Upcoming Projects Future Concepts TIMS Program Timeline 1993 TMC
Monitoring a freeway network in real time using single-loop detectors. Xiaoping Zhang*, Yinhai Wang and Nancy L. Nihan
Int. J. Vehicle Information and Communication Systems, Vol. 1, Nos. 1/2, 2005 119 Monitoring a freeway network in real time using single-loop detectors Xiaoping Zhang*, Yinhai Wang and Nancy L. Nihan Department
IFS-8000 V2.0 INFORMATION FUSION SYSTEM
IFS-8000 V2.0 INFORMATION FUSION SYSTEM IFS-8000 V2.0 Overview IFS-8000 v2.0 is a flexible, scalable and modular IT system to support the processes of aggregation of information from intercepts to intelligence
Final Report - HydrometDB Belize s Climatic Database Management System. Executive Summary
Executive Summary Belize s HydrometDB is a Climatic Database Management System (CDMS) that allows easy integration of multiple sources of automatic and manual stations, data quality control procedures,
How To Manage Big Data For The Ohio Department Of Transportation
Big Data & Asset Management Ohio Planning Conference 2014 John R. Kasich, Governor Jerry Wray, Director Ohio Department of Transportation Big Data: Asset Management Andrew Williams, Administrator Office
Crash data may not contain complete information, some elements may be unknown
Crash Data Introduction: Crash data are information that comes from a reportable crash. A reportable crash according to Title 75, Pennsylvania Consolidated Statutes, Section 3746(a) is: an incident that
SF Bay Area Transportation Management Systems: An Innovative DOT and MPO Partnership
SF Bay Area Transportation Management Systems: An Innovative DOT and MPO Partnership Radiah Victor Principal, Operations Metropolitan Transportation Commission Oakland, California ITS CA Annual Meeting
Advanced Enterprise Work and Asset Management for Performance-Driven Utilities
Advanced Enterprise Work and Asset Management for Performance-Driven Utilities Asset & Resource Management (ARM) 2 CGI s Asset & Resource Management (ARM) 2 suite is a solution designed to streamline the
Integrating GIS-Based Videolog and Asset Data With Commonly Used Systems Provides Major Benefits With Minimal Effort
Integrating GIS-Based Videolog and Asset Data With Commonly Used Systems Provides Major Benefits With Minimal Effort Presented By: Debbie Burns, GIS Manager Michael Nieminen, Project Manager Introduction
Paper 064-2014. Robert Bonham, Gregory A. Smith, SAS Institute Inc., Cary NC
Paper 064-2014 Log entries, Events, Performance Measures, and SLAs: Understanding and Managing your SAS Deployment by Leveraging the SAS Environment Manager Data Mart ABSTRACT Robert Bonham, Gregory A.
INVESTIGATION OF ASIM 29X, CANOGA, RTMS, SAS-1, SMARTSENSOR, TIRTL & OTHER SENSORS FOR AUTOMATIC VEHICLE CLASSIFICATION
INVESTIGATION OF ASIM 29X, CANOGA, RTMS, SAS-1, SMARTSENSOR, TIRTL & OTHER SENSORS FOR AUTOMATIC VEHICLE CLASSIFICATION RESEARCH NEED This research will add to the national and local state-of-the-art on
22 nd ITS World Congress Towards Intelligent Mobility Better Use of Space. GPS 2: Big Data The Real Value of Your Social Media Accounts
22 nd ITS World Congress Towards Intelligent Mobility Better Use of Space GPS 2: Big Data The Real Value of Your Social Media Accounts October 7, 2015 Kenneth Leonard Director, Intelligent Transportation
Impact of Connected Automated Vehicles on Traffic Management Centers (TMCs)
Impact of Connected Automated Vehicles on Traffic Management Centers (TMCs) Automated Vehicles Symposium 2015 Breakout Session Impact of Connected and Automated Vehicles on Traffic Management Systems and
Data Cleansing for Remote Battery System Monitoring
Data Cleansing for Remote Battery System Monitoring Gregory W. Ratcliff Randall Wald Taghi M. Khoshgoftaar Director, Life Cycle Management Senior Research Associate Director, Data Mining and Emerson Network
I-29 Corridor ITS Architecture and Systems Engineering Analysis
430 IACC Building Fargo, ND 58105 Tel 701-231-8058 Fax 701-231-1945 www.ugpti.org www.atacenter.org I-29 Corridor ITS Architecture and Systems Engineering Analysis Technical Memorandum December 2001 Prepared
Data Management Implementation Plan
Appendix 8.H Data Management Implementation Plan Prepared by Vikram Vyas CRESP-Amchitka Data Management Component 1. INTRODUCTION... 2 1.1. OBJECTIVES AND SCOPE... 2 2. DATA REPORTING CONVENTIONS... 2
Chapter 3 - GPS Data Collection Description and Validation
Chapter 3 - GPS Data Collection Description and Validation The first step toward the analysis of accuracy and reliability of AVI system was to identify a suitable benchmark for measuring AVI system performance.
BENEFITS OF AUTOMATING DATA WAREHOUSING
BENEFITS OF AUTOMATING DATA WAREHOUSING Introduction...2 The Process...2 The Problem...2 The Solution...2 Benefits...2 Background...3 Automating the Data Warehouse with UC4 Workload Automation Suite...3
Traffic Monitoring Guide May 1, 2001. Traffic Volume Monitoring
Traffic Volume Monitoring SECTION 3 CONTENTS Section Page CHAPTER 1 INTRODUCTION...3-1 Traffic Volume Data Collection...3-1 Objectives of the Traffic Volume Monitoring Program...3-2 Organization of This
ASSET MANAGEMENT OF INTELLIGENT TRANSPORTATION SYSTEMS: METHODOLOGY FOR SELECTING THE OPTIMAL PLATFORM FOR YOUR AGENCY
0 0 0 ASSET MANAGEMENT OF INTELLIGENT TRANSPORTATION SYSTEMS: METHODOLOGY FOR SELECTING THE OPTIMAL PLATFORM FOR YOUR AGENCY TaheraAnjuman Clemson University Clemson, South Carolina (0) - [email protected]
Business Value Reporting and Analytics
IP Telephony Contact Centers Mobility Services WHITE PAPER Business Value Reporting and Analytics Avaya Operational Analyst April 2005 avaya.com Table of Contents Section 1: Introduction... 1 Section 2:
Nevada DOT s s IRWIN Project
Nevada DOT s s IRWIN Project Integrated Right of Way Information Network Presented By: Eric Warmath, NDOT GIS Coordinator and IRWIN Coordination Team Member Major Project Goals IRWIN Modular Design NDOT
Simulating Traffic for Incident Management and ITS Investment Decisions
1998 TRANSPORTATION CONFERENCE PROCEEDINGS 7 Simulating Traffic for Incident Management and ITS Investment Decisions MICHAEL D. ANDERSON AND REGINALD R. SOULEYRETTE UTPS-type models were designed to adequately
Tracking System for GPS Devices and Mining of Spatial Data
Tracking System for GPS Devices and Mining of Spatial Data AIDA ALISPAHIC, DZENANA DONKO Department for Computer Science and Informatics Faculty of Electrical Engineering, University of Sarajevo Zmaja
Study and Calculation of Travel Time Reliability Measures
Center for Applied Demography & Survey Research University of Delaware Study and Calculation of Travel Time Reliability Measures by David P. Racca Daniel T. Brown Prepared for the Delaware Department of
Using Cloud-Computing to Promote Asset Management Best Practices A Ministry of Transportation Ontario Case Study
Using Cloud-Computing to Promote Asset Management Best Practices A Ministry of Transportation Ontario Case Study Benjamin Ong Fugro Roadware 2505 Meadowvale Blvd. Mississauga, ON L5N 5S2 Ph: (905) 567-2870
Application of GIS in Transportation Planning: The Case of Riyadh, the Kingdom of Saudi Arabia
Application of GIS in Transportation Planning: The Case of Riyadh, the Kingdom of Saudi Arabia Mezyad Alterkawi King Saud University, Kingdom of Saudi Arabia * Abstract This paper is intended to illustrate
A304a: Understanding User Needs for Field Management Stations Part 1 Object Definitions for Signal System Masters (SSM) Based on NTCIP 1210 Standard
A304a: Understanding User Needs for Field Management Stations Part 1 Object Definitions for Signal System Masters (SSM) Based on NTCIP 1210 Standard Table of Contents Introduction/Purpose... 2 SSM User
Functional Requirements Document -Use Cases-
EUROPEAN COMMISSION DIRECTORATE GENERAL JRC JOINT RESEARCH CENTRE Cyber-security & New Technologies for Combating Fraud (CSCF) Institute for the Protection and Security of the Citizen (IPSC) EYE IN THE
Advanced Metering Information Systems
e Executive Brief: Advanced ing Information s Introduction e s mission is to help utilities maximize the value of their advanced metering infrastructure ( AMI ) and fixed network 1 investments. Most utilities
The TomTom Manifesto Reducing Congestion for All Big traffic data for smart mobility, traffic planning and traffic management
The TomTom Manifesto Reducing Congestion for All Big traffic data for smart mobility, traffic planning and traffic management Ralf-Peter Schäfer Fellow & VP Traffic and Travel Information Product Unit
An Oracle White Paper March 2014. Best Practices for Real-Time Data Warehousing
An Oracle White Paper March 2014 Best Practices for Real-Time Data Warehousing Executive Overview Today s integration project teams face the daunting challenge that, while data volumes are exponentially
California PATH, University of California at Berkeley, Building 452, Richmond Field Station 1357 South 46 th Street, Richmond, CA 94804
1. Report No. CA14-2146B 2. Government Accession No. 3. Recipient s Catalog No. 4. Title and Subtitle Executive Summary for the UC Berkeley, California PATH s Augmented Speed Enforcement Project 5. Report
EcOS (Economic Outlook Suite)
EcOS (Economic Outlook Suite) Customer profile The International Monetary Fund (IMF) is an international organization working to promote international monetary cooperation and exchange stability; to foster
Real-time customer information data quality and location based service determination implementation best practices.
White paper Location Intelligence Location and Business Data Real-time customer information data quality and location based service determination implementation best practices. Page 2 Real-time customer
Geocoding in Law Enforcement Final Report
Geocoding in Law Enforcement Final Report Geocoding in Law Enforcement Final Report Prepared by: The Crime Mapping Laboratory Police Foundation August 2000 Report to the Office of Community Oriented Policing
GEOGRAPHIC INFORMATION SYSTEMS (GIS): THE BEDROCK OF NG9-1-1
GEOGRAPHIC INFORMATION SYSTEMS (GIS): THE BEDROCK OF NG9-1-1 THE TIME IS NOW FOR PSAPS AND REGIONAL AGENCIES TO TAKE ADVANTAGE OF THE ACCURATE GEOSPATIAL DATABASES THAT WILL BE KEY TO NEXT GENERATION EMERGENCY
BANGKOK TRAFFIC MONITORING SYSTEM
BANGKOK TRAFFIC MONITORING SYSTEM Sorawit NARUPITI Associate Professor Transportation Research Laboratory, Department of Civil Engineering, Chulalongkorn University Phayathai Road, Bangkok 10330 Thailand
NTT DATA Big Data Reference Architecture Ver. 1.0
NTT DATA Big Data Reference Architecture Ver. 1.0 Big Data Reference Architecture is a joint work of NTT DATA and EVERIS SPAIN, S.L.U. Table of Contents Chap.1 Advance of Big Data Utilization... 2 Chap.2
SOLUTION COMMUNICATIONS
SOLUTION COMMUNICATIONS Infrastructure Management Your Complete communications Solution Perhaps more than any other high-technology sector, the communications industry is experiencing unprecedented technological
Short-Term Forecasting in Retail Energy Markets
Itron White Paper Energy Forecasting Short-Term Forecasting in Retail Energy Markets Frank A. Monforte, Ph.D Director, Itron Forecasting 2006, Itron Inc. All rights reserved. 1 Introduction 4 Forecasting
EXECUTIVE SUMMARY. Syracuse Metropolitan Area Intelligent Transportation Systems Strategic Plan
Syracuse Metropolitan Area Intelligent Transportation Systems Strategic Plan Final Report EXECUTIVE SUMMARY Prepared for New York State Department of Transportation & Syracuse Metropolitan Transportation
SECTION 16 TRAFFIC/SAFETY SECTIONS 16.1, 16.2 AND 16.3 ARE UNDER DEVELOPMENT
SECTION 16 TRAFFIC/SAFETY SECTIONS 16.1, 16.2 AND 16.3 ARE UNDER DEVELOPMENT 16.1-1 16.4 Intelligent Transportation Systems Introduction The ITS Engineering Unit is responsible for the design of all ITS
A RFID Data-Cleaning Algorithm Based on Communication Information among RFID Readers
, pp.155-164 http://dx.doi.org/10.14257/ijunesst.2015.8.1.14 A RFID Data-Cleaning Algorithm Based on Communication Information among RFID Readers Yunhua Gu, Bao Gao, Jin Wang, Mingshu Yin and Junyong Zhang
The Data Warehouse ETL Toolkit
2008 AGI-Information Management Consultants May be used for personal purporses only or by libraries associated to dandelon.com network. The Data Warehouse ETL Toolkit Practical Techniques for Extracting,
Role of Analytics in Infrastructure Management
Role of Analytics in Infrastructure Management Contents Overview...3 Consolidation versus Rationalization...5 Charting a Course for Gaining an Understanding...6 Visibility into Your Storage Infrastructure...7
Comparing Microsoft SQL Server 2005 Replication and DataXtend Remote Edition for Mobile and Distributed Applications
Comparing Microsoft SQL Server 2005 Replication and DataXtend Remote Edition for Mobile and Distributed Applications White Paper Table of Contents Overview...3 Replication Types Supported...3 Set-up &
Module 2.1. Page 1 of 10. Slide 1
Slide 1 Module 2: Traffic Incident Management Framework & Resources Lesson 2.1 Regional Transportation Management Centers This course was developed by the Baltimore Regional Operations Coordination Committee
Delaware Department of Transportation Transportation Management Program
Delaware Department of Transportation Transportation Management Program Delmarva Freight & Goods Movement Task Force Meeting December 4, 2013 Overview ITMS Background Relationship to Freight and Goods
How To Handle Big Data With A Data Scientist
III Big Data Technologies Today, new technologies make it possible to realize value from Big Data. Big data technologies can replace highly customized, expensive legacy systems with a standard solution
A SPECIAL APPLICATION OF A VIDEO-IMAGE SYSTEM FOR VEHICLE TRACKING AND SPEED MEASUREMENT
A SPECIAL APPLICATION OF A VIDEO-IMAGE SYSTEM FOR VEHICLE TRACKING AND SPEED MEASUREMENT Zong Tian (Corresponding Author) Texas Transportation Institute The Texas A&M University System College Station,
Associate Professor, Department of CSE, Shri Vishnu Engineering College for Women, Andhra Pradesh, India 2
Volume 6, Issue 3, March 2016 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Special Issue
DEVELOPMENT OF INTEGRATED HIGHWAY MANAGEMENT SYSTEM IN KOREA
DEVELOPMENT OF INTEGRATED HIGHWAY MANAGEMENT SYSTEM IN KOREA Chunjoo YOON Researcher Highway Research Department Korea Institute of Construction Technology 2311, Daehwa-Dong, Ilsan-Gu, Goyang-Si, Gyeonggi-Do,
2. SYSTEM ARCHITECTURE AND HARDWARE DESIGN
Design and Evaluation of a Wireless Sensor Network for Monitoring Traffic Yuhe Zhang 1, 2, Xi Huang 1, 2, Li Cui 1,, Ze Zhao 1 Tel: +86-10-6260 0724, Fax: +86-10-6256 2701 1 Inst. of Computing Technology,
Global Value 7. Productivity Suite for GammaVision. Optimizing Gamma Spectrometry Processes through Secure Data Management and Measurement Automation.
Productivity Suite for GammaVision Optimizing Gamma Spectrometry Processes through Secure Data Management and Measurement Automation. Global Value was designed to transform the standard GammaVision spectroscopy
Please Note: Temporary Graduate 485 skills assessments applicants should only apply for ANZSCO codes listed in the Skilled Occupation List above.
ANZSCO Descriptions This ANZSCO description document has been created to assist applicants in nominating an occupation for an ICT skill assessment application. The document lists all the ANZSCO codes that
Looking Beyond Data Synchronization for Mission Critical GIS Data
Looking Beyond Data Synchronization for Mission Critical GIS Data Table of Contents Preface... 2 Background Information... 3 Introduction... 4 Area of Focus #1: Locally Authoritative GIS Data Development
Developing Fleet and Asset Tracking Solutions with Web Maps
Developing Fleet and Asset Tracking Solutions with Web Maps Introduction Many organizations have mobile field staff that perform business processes away from the office which include sales, service, maintenance,
THE DATA WAREHOUSE ETL TOOLKIT CDT803 Three Days
Three Days Prerequisites Students should have at least some experience with any relational database management system. Who Should Attend This course is targeted at technical staff, team leaders and project
Real Time Remote Monitoring over Cellular Networks. Wayne Chen Marketing Specialist
Real Time Remote Monitoring over Cellular Networks Wayne Chen Marketing Specialist Introduction For distribution infrastructures located in remote, isolated areas, unmanned monitoring systems have long
RTMS Solutions. Detection solutions to fit your city s needs.
RTMS Solutions Detection solutions to fit your city s needs. Keeping your daily commute free of congestion - and maintenance. Behind the scenes in cities all over the world, RTMS radar detection solutions
