INTEGRATION OF CRASH DATA INTO A GEOGRAPHIC INFORMATION SYSTEM. Technical Memorandum

Transcription

1 INTEGRATION OF CRASH DATA INTO A GEOGRAPHIC INFORMATION SYSTEM July 2009

2 INTEGRATION OF CRASH DATA INTO A GEOGRAPHIC INFORMATION SYSTEM Prepared for Prepared by University of Florida Department of Urban and Regional Planning Contact: Ilir Bejleri, ilir@ufl.edu, Tel ext. 432 July

3 Executive Summary The purpose of this project is to develop processes and tools to allow staff develop and maintain a Geographic Information System (GIS) crash database for all crashes in Orange, Osceola and Seminole counties, as well as to explore options for sharing the data with partner agencies. The first step in this process was the assessment of candidate crash data sources such as Florida Department of Highway Safety and Motor Vehicles (DHSMV), Prison Rehabilitative Industries and Diversified Enterprises Inc. (PRIDE) and Florida Department of Transportation (FDOT). DHSMV was chosen as the most suitable source for long form crashes due to simplicity of the data structure and friendly data format, good data documentation, and availability of both data and scanned images, all free of charge. PRIDE was chosen as the source for short form data because it is the only reliable source that can provide short form crash regional data (for a fee) in one consistent standard format. The assessment of hardware and software concluded that Metroplan Orlando does have the supporting technologies to develop and maintain a crash database with data obtained from DHSMV and PRIDE. For optimal results and greater productivity, staff can benefit from improved computer specs especially by replacing a single monitor with dual monitors. This project developed custom tools to automate merging of the PRIDE and DHSMV data into one unified crash database. Custom tools are also made available for automatic and interactive geocoding using the GIS street centerlines of each county as the spatial reference layer for geocoding. The automatic geocoding match rate reached 70% after improvements of the street names tables. It is estimated that about 30% of the crash records may have to be geocoded interactively. This amounts to about 60 crashes a day or 300 crashes per week for all three counties. With proper training and the help of local knowledge, as an on-going process, this task can be accomplished in 2-3 days per week by one full time staff or in about 5 days per week by one part time staff. Hiring one part time intern is recommended as the most economical strategy. A computer and a copy of the ArcView software should also be provided for the intern or the new staff member. The Metroplan staff was trained to map and maintain the crash data independently. Data sharing with the partner agencies can be achieved via ftp or web site download. ESRI s personal geodatabase format is the primary recommended format for data distribution format due to its suitability for the crash data structure, compatibility with MS Access, and the predominance of ESRI GIS software among the partner agencies in the area. If access to scanned images is required by some partner agency, can consider distribution on DVDs. However Metroplan Orlando would also consider its available resources to accomplish this in case of high demand for this information. Required resources would include the staff time to burn the DVDs, cost of media, cost of mail, and distribution frequency. Concerns about data privacy requirements should be cleared before distribution. 3

4 A second avenue to help partner agencies is to provide access to GIS crash data analysis and reporting tools via the internet. would host and maintain a webbased crash data analysis and reporting system which would be accessed by partner agencies using a web browser and a secure login mechanism. This is a more expensive upfront solution but more comprehensive and useful for partner agencies. Recommended hardware and software configuration specs are provided in the last section of this report. 4

5 Table of Contents 1. INTRODUCTION ASSESSMENT OF USER INTERFACE AND DATA MANAGEMENT CRASH DATA GIS DATA SOFTWARE & USER INTERFACE HARDWARE CONCLUSION INTERFACE DEVELOPMENT TOOL FOR CREATING THE UNIFIED DATABASE TOOLS TO GEOCODE CASH DATA TRAINING STAFFING CONCLUSION DATA SHARING DATA DISTRIBUTION Data Structure Data Formats Distribution Methods Data Documentations (metadata) Update Frequency Data Privacy Issues Scanned Crash Reports Conclusion ACCESS TO ANALYSIS AND REPORTING Web-Based Crash Data Analysis and Reporting System Configuration Recommendations REFERENCES: APPENDIX A EXAMPLES OF ANALYSIS AND REPORTING

6 1. Introduction The Safe, Accountable, Flexible, and Efficient Transportation Equity Act: A Legacy for Users (SAFETEA: LU) law emphasized the need for crash data systems with the ability to perform safety problem identification and countermeasure analysis on all public roads. has recognized the importance of traffic safety and development of strategies to reduce pedestrian, bicycle and vehicle fatalities and injuries. In 2006, Metroplan conducted a feasibility study on establishing a crash database. Among other findings, this study confirmed the need of various agencies to have access to geocoded crash data and related crash attributes in order to improve their work on traffic safety. This project is the continuation of the Metroplan efforts to improve access to crash data. The purpose of this project is to develop processes and tools to allow Metroplan Orlando staff develop and maintain a GIS crash database for all crashes in Orange, Osceola and Seminole counties. More specifically this project will enable the Metroplan staff to obtain, process, structure, geocode and analyze the crash data, as well as share the data with the Metroplan partners. This will complement the existing service that Metroplan provides to partners in the campaign to make roadway safer. In order to accomplish its objectives this project focused on three main objectives: (a) Assessment of crash data sources and technical capacities to maintain a crash database; (b) Development of automated data management and mapping tools and (c) Options for data sharing mechanisms. This document describes accomplishments of these objectives. 2. Assessment of User Interface and Data Management There are four parts included in this assessment: crash data, GIS data, software & interface and hardware. 2.1 Crash Data Figure 1 illustrates the process of crash data collection flow in the State of Florida. Law enforcement agencies record each crash using the Florida standard crash report form. A copy of each crash report in the state of Florida is sent to DHSMV which serve as the state repository for crash data. Except for Florida Highway Patrol (FHP) the rest of Florida law enforcement agencies send to DHSMV paper copies of crash reports. In turn, DHSMV sends the paper reports to PRIDE which enters the data into a digital database (including scanning every single report) and sends back to DHSMV the digital records. DHSMV sends a copy of the digital long forms and images to FDOT and by request to any other agency in the state that may request the data. 6

7 Few notes: PRIDE doesn t enter the short form data into the database (except for few index fields which are not useful for analysis). The data collected by FHP are transmitted to DHSMV digitally and are not sent to PRIDE. Law Enforcements FHP OPD Long and short paper forms HSMV Long and short paper forms PRIDE Digital long forms, all scanned images Figure 1 Crash Data Collection Flow in the State Of Florida. Based on this data flow, the crash data for the area can be obtained from three potential sources: DHSMV, PRIDE or FDOT. DHSMV provides free of charge long form crash data and scanned images and FHP short form data and images monthly through a memorandum of understanding with. The data is delivered to on DVD. The DHSMV data is distributed in 9 comma delimited text files. These files are standard statewide. Figure 2a below shows the names of these files. The data is well documented through a data dictionary. The data latency is typically about 6 weeks. PRIDE provides short form data (collected by all agencies minus FHP) and images for a fee because short forms are not part of the contract that PRIDE has with DHSMV. PRIDE delivers the data to on CDROM. The PRIDE data is distributed in MS Access 2003 format. It is organized in 20 different tables. There is no data dictionary but the data field names for the most part are easy to understand. Figure 2b shows the PRIDE data tables. PRIDE data are also standard statewide. The data delivered by PRIDE is about 60 days old. 7

8 a. DHSMV crash data text files b. PRIDE MS Access crash data tables Figure 2 DHSMV and PRIDE Crash Data. Both DHSMV and PRIDE data are keyed in a digital database from paper form by PRIDE. FDOT uses long form crash data through its Crash Analysis and Reporting System (CARS). FDOT receives the crash data from DHSMV and restructures them in a suitable form for the FDOT business process. The FDOT data structure matches neither PRIDE nor the DHSMV structure and the data is more difficult to understand and work with. The data structure lacks user-friendly documentation. Acquiring scanned images requires special arrangement since they are not easily accessible in bulk. The FDOT has started to include crashes on local roads, however only for long forms. The review of the three potential data sources suggests that DHSMV and PRIDE represent two chosen data sources that would fulfill need to obtain and maintain a crash database. Although their data structures are different, they are not difficult to understand in order to unify into a single format. In addition, the images of the scanned reports are available from both sources. To map and analyze the crash data coming from more than one source in this case DHSMV and PRIDE - a single unified structure of the crash data is required. Two main approaches are considered: (a) develop a new structure that doesn t match any of the data sources and convert each input data source to the unified structure or (b) select one of the sources to serve as the data structure and convert the rest of the 8

9 sources to the selected structure. The second solution was chosen for this project the DHSMV data structure was chosen as the unified data structure for several reasons: DHSMV data is distributed in a structure that is compact, with a smaller number of tables and the relationship among tables are simpler to establish. The DHSMV format is industry standard comma delimited. DHSMV data structure is well documented by the DHSMV data dictionary distributed with the data. In the future, it is expected that DHSMV will start distributing the short form crash data from all agencies. The short form data will be distributed in the same current DHSMV structure. At that point there will be one single source for crash data DHSMV. The PRIDE data will not be needed any longer DHSMV is the official Florida agency responsible for collecting and distributing the crash data. The diagram in Figure 3 below shows the unified data structure diagram: Figure 3 Unified Data Structure Diagram A detailed description of tables and relationships is provided in the user s manual of the tools developed for this project. Note that the table named Spatial_Location is not part of the DHSMV data structure. This table is proposed as part of the unified structure to store the GIS crash points that will be created through geocoding. 2.2 GIS Data The necessary GIS data required for mapping and spatial analysis of traffic crashes are street centerlines, and alternative street names. doesn t 9

10 maintain its own GIS data which means that has to rely on other sources, possibly from its partner network. Two options are explored: (a) Use the GIS data maintained by GIS departments of each of the three counties, Orange, Seminole and Osceola or (b) Use data from commercial sources purchased by FDOT. Both options would be free of charge for Metroplan use. The local centerlines use a single centerline model which means that each road (except for interstates) is represented by a single centerline. This model is suitable for mapping the crash data in Florida because unfortunately the location information on current crash data is not very accurate to benefit by a more detailed street network representation. The commercial data, currently from vendor TeleAtlas, use a dual centerline model. This is a more detailed than a single centerline model but the crash data at their present state may not be able to take advantage of this model due to its limited information. In addition, the dual centerline model, although desirable, is more complex to use for analysis and reporting. Last, the FDOT may change the vendor in the future which will require that all the crash data that were mapped on one vendor s street centerlines network has to be moved or remapped into the network of the new vendor. In fact, just recently the FDOT completed a contract to acquire NAVTEQ base-map and is planning to provide it free of charge to public agencies via their web-site. The local street centerlines are not perfect. There may be spatial and attribute inaccuracies but commercial data are not immune to these problems either. Another problem with local data was the lack of alternative street names; however this problem can be corrected. The TeleAtlas data does have alternative street names; however the alternate names are provided in the GIS database by multiplication of street segments that have many names. This structure, although workable for geocoding, can present problems for GIS analysis if users are unaware of it. The local street centerlines have differences in their table structure. They work well for analysis within individual counties but not when a regional analysis is required. However, Lynx has merged all the centerlines into one single layer which can be consider for use in case such analysis is required. The Lynx version has preserved the original street identification system which can allow transferring of crash points to the Lynx structure by ID association. Last, given the mission of to share the data with its partners, the mapped crashes for partner county agencies will be of use when mapped using the local data. In conclusion, the local data was recommended as the most suitable choice for GIS street centerlines and alternate street names tables. In the future, if and when the three counties may choose to adopt the commercial street network provided free of charge by FDOT, can consider changing the GIS streets reference system. 10

11 2.3 Software & User Interface Two main software packages are needed for management and mapping of crash data: (a) database management and (b) GIS. has access to two options for database management software: MS Access and MS SQL Server. The MS Access is recommended for single user and is much friendlier than MS SQL Server. It is directly compatible with the ESRI s personal geodatabase spatial format. MS Access comes equipped with tools for queries, reporting and charting. MS SQL Server requires a special and more expensive interface to handle GIS data and usually requires database administration experience. GIS software of choice for Metroplan is ArcGIS. Metroplan already has an ArcView license and its staff members that use GIS in their work are currently using ArcGIS. Both MS Access and ArcGIS have tools respectively for data management, geocoding and spatial analysis but they are general tools not customized for crash data mapping and analysis. Although the data can be imported in MS Access using it s out of the box interface and the data can be mapped in GIS using its standard interface, working with crash data involve many repetitive time-consuming processes that can benefit from custom tools and user interfaces. These tools are developed and will be described in the next sections. 2.4 Hardware A typical PC configuration can run both MS Access and ArcGIS. The Metroplan Orlando staff has been able to perform typical tasks for data management and geocoding using current hardware and software. In case of desire to increase efficiency and productivity it is recommended that Metroplan staff working on geocoding should have dual monitor of size of 20 or higher. A computer with 2GB of RAM would do the job but 4GB would be recommended. A dual core CPU processor is recommended. Hard disk space nowadays is inexpensive. A hard disk of with 250GB of space or higher would be able to store many years of crash data. The database storage should be placed on the network for easy backup and wider access. In case of budget shortage the priority for upgrades should focus on dual monitor and RAM increase. 2.5 Conclusion The current technologies present at are sufficient for building and maintaining a GIS crash database. An additional copy of ArcView software may be required if new staff is hired to support geocoding needs. To learn about the requirements for sharing the data with interested parties see recommendations in the Data Sharing section below. 11

12 3. Interface Development The overall process from obtaining crash data to the display of analysis results on a map goes through several distinct steps that involve importing and merging the crash data into a unified database, geocoding of crashes and use of mapped crashes for spatial analysis. This process is illustrated in the Figure 4 below: Figure 4 Crash Database Creation Overview There are four main objectives for this task: (a) Develop an automated process to create a unified crash database, (b) Create tools to geocode the crash data (c) Train the current Metroplan staff on developing and geocoding the crash database and (d) Estimate staffing level and time necessary to geocode unmatched records by the automated process as well as to complete and maintain the crash database. 3.1 Tool for Creating the Unified Database As described in the first section, the unified database that matches that of the DHSMV consists of nine related tables. The figure 5 below illustrates the unified data structure showed in MS Access format. 12

13 a. Unified data structure b. Florida Crash Data Import Utility Figure 5 Unified Data Structure and Import Tool This data structure can be created by using the Florida Crash Data Import Utility, a tool developed by the University of Florida for this purpose. The tool has the capability to import DHSMV comma delimited text files, MS Access PRIDE data as well to extract data from the database for use as needed. The interface of the tool is illustrated in Figure 5b above. A detailed operation of the tools is provided in the user s manual. 3.2 Tools to Geocode Cash Data Several tools are provided to assist in automatic and interactive geocoding of crashes. The tools are grouped in a custom ArcGIS package called GeoCrashTools. The main toolbar of the interface is shown in figure 6 below. 13

14 Data Preparation Geocoding Crash Analysis Figure 6 GeoCrashTools Toolbar The GeoCrashTools include three tool categories (a) Data Preparation Tools (b) Geocoding Tools and (c) Analysis Tools Data Preparation Tools: Include tools for building supporting files for geocoding and crash analysis as well as tools for processing and preparing the crash data for geocoding. This include categorizing crash data by location type, tagging crashes that involve pedestrians or bicycles, correcting city codes or names, formatting the crash address as required by geocoding, tagging crashes that occurred in parking lots or private property etc. Geocoding Tools: These tools perform automatic (batch) geocoding, interactive and manual geocoding. This set includes a tool for creating a geocoding dictionary for frequently misspelled street names in order to improve geocoding match rate. Analysis Tools: Include tools that summarize crashes by street intersections and street segments to determine high crash locations. Included is a tool that can assist in ranking street segments or proposed projects based on the crash injury severity. Also, if traffic volume is available these tools can determine the crash rate which can be used to compare high crash locations. Another tool called Aggregation breakdown can perform comparative analysis of high crash location by considering different factors involved such as cash injury severity, weather, road conditions etc. 14

15 3.3 Training Hands-on training was provided to Metroplan Staff on using the tools above. The Metroplan staff has now the knowledge and the experience to build and maintain independently the crash database for Metroplan area. In addition the University of Florida (UF) team provided assistance in developing the initial crash database and all the required GIS data for mapping and analysis. A user s manual is provided to Metroplan staff with details on the operations of the tools described above. 3.4 Staffing The necessary staffing to maintain a GIS crash database for the area is required to accomplish two main groups of tasks: (a) Monthly import and automatic geocoding of crashes and (b) Geocoding of crashes that do not match automatically. The automated process is expected to be performed once a month and it make take one to two days to be completed by a full time staff member that has proper training and experience. The geocoding of unmatched crashes is more time consuming. Testing with 2008 crash data from DHSMV and PRIDE shows that about 70% of crashes geocode automatically when a good alternative street names table is available. This number will increase by improving the alternative street names table and by expanding the crash dictionary with typical errors in crash addresses as well as by continuing to customize the classification files with entries that are typically encountered during geocoding. The rest of the records, typically 25-30%, will require staff attention. Below is shown the number of crashes per month based on a 5-year average for both short and long form data for area. These numbers are provided by PRIDE. County Long Forms Short Forms Total Orange Osceola Seminole Total Table 1 Average number of crashes per month (Source: PRIDE 5 Year Average) Based on these numbers gets an average of 60,000 crashes a year for both long and short forms. Roughly 12-14% of these crashes are parking lot or private property crashes and will not be included in geocoding. Based on these numbers and an estimated conservative 30% unmatched rate, there will be about 60 crashes/day or 300 crashes/week that may require interactive geocoding. With proper training and local knowledge one can geocode these crashes in 2-3 days working full time or in about one week working part time. If parking lot and private property crashes are included consider 70 crashes/day or 350 crashes /week. 15

16 One strategy for staffing consideration is to have the current experienced staff handle the automatic process and hire part-time interns to handle the unmatched records. This may be the most economical solution but some downsides should be considered: interns would need to be trained for a couple of weeks. Because the interns are going to work only for a limited amount of time, perhaps 3-4 months, it is also recommended to overlap them for a couple of weeks to allow sufficient time for knowledge transfer. This strategy that expects the current staff to take on the automatic process, assumes that the current staff can afford to use part of his time for the automated process. It should be noted that in addition to handling the automatic process, the current staff may also have to develop reports and/or conduct analysis using the crash data which would require additional time. Fulfilling request for data, analysis and reports for partners will require additional time. Another option may be to hire a full time intern or a full time staff member to accomplish both the automatic process and the interactive geocoding. This will free the current staff from the tasks of day to day crash database maintenance to allow more time for analysis and reporting. In either case, increased experience in interactive geocoding and local knowledge can contribute to increased productivity which can reduce the estimated processing time recommended above. 3.5 Conclusion has the hardware, the supporting software and the custom database and mapping tools as well as training to maintain a GIS crash database for the Orange, Seminole and Osceola counties. Metroplan needs to weight options for staffing that range from handling the entire workload by the current staff (if current staff can afford to spent few days each week on crashes) or hire a dedicated part time or full time new staff or intern. 16

17 4. Data Sharing As highlighted in the report Feasibility of Developing and Maintaining an Automated Vehicle Crash Data base in the Orlando Urban Area there are many partner agencies in the area that are interested in having access to timely GIS crash data. Two levels of data sharing are discussed below: (a) Providing the interested parties with a copy of the GIS crash data which is referred here as Data Distribution and (b) Creating a system that gives interested users access to GIS crash data analysis and reporting rather than the data per se. This option is referred here as Access to Analysis and Reporting. Discussion and recommendations are provided below for both options. 4.1 Data Distribution Sharing the data is defined here as providing a copy of the data to the interested parties. This assumes that the interested parties can use the data for their needs and have the resources and capacities to do so. Figure 7 illustrate the crash data flow and distribution to partners through. will build and maintain the GIS Crash Database by cleaning, unifying and geocoding the crash data from PRIDE and DHSMV. The crash data and the GIS crash points will be made available for download to partner agencies. The scanned images could be distributed via DVD to interested partners. 17

18 PARTNER AGENCIES: can download data Law Enforcement Engineering Other FHP OPD INTERNET INTERNET Long and short paper forms METROPLAN ORLANDO Download, DVD GIS Crash Database (x y, data, images) METROPLAN ORLANDO Clean, Unify, Geocode Digital long forms and images Digital short forms and images HSMV Long and short paper forms PRIDE Digital long forms, all scanned images Figure 7 Crash Data Flow and Distribution through Sharing the data involves establishing data distribution structure, formats, distribution methods, documentation and update frequencies. Data privacy issues should be also considered. This section provides recommendations for Metroplan Orlando for sharing or distributing the crash data to the interested partners Data Structure The 9-table data structure is already a stable structure expected to continue to be provided by DHSMV. The tables can be linked together via primary keys already included in the data. This table structure is sufficient for the 18

19 purposes of analysis and reporting. Using this same structure for data management and for distribution will minimize the time for Metroplan Orlando staff to fulfill data requests. The current structure fulfills goals and there is no compelling argument at present that would dictate a need for modification of this structure. Therefore, it is recommended that uses this structure for data distribution. It should be noted that a 10 th table or spatial layer containing the spatial location of crash points should be added to the distribution database. See data formats below for details on this spatial table/layer Data Formats The crash data include two components: crash point spatial data (x,y coordinates of crash locations) and tabular crash data attributes associated with the crash event, drivers, vehicles, pedestrians etc. The following formats are considered: Point Spatial Data: (a) ESRI shapefile this is the most widespread industry format for GIS sharing data. Most, if not all, GIS software packages have capabilities to read shapefiles. The potential problem with this format is its relatively fragile data structure. A shapefile, although seemly a single file, is in fact made of a minimum of the three related files (.shp,.shx and.dbf). The shapefile may get easily corrupted if missing one or more of its components. This problem can be minimized by providing the shapefile in a compressed format (zip) as a single file. (b) A tabular structure in either dbf or excel format. The table will contain two fields for the x,y coordinates and the DHSMV report number. The advantage of this format is the ability to be imported by a very large number of software packages. The downside of this format is that the tabular data needs to be converted to a spatial format. This can be accomplished by any GIS software of choice. For ArcGIS users, tabular data that contain x,y coordinates of point data can be converted to spatial format by using function Add XY Data that comes standard with the basic ArcView license. (c) a third option is to provide the crash data in a Personal Geodatabase format. This format is more robust and works well for data distribution. One downside of this format is its limited compatibility. It is a format designed to work with the ArcGIS software and there are only a few other GIS packages that can read it directly. However, the personal geodatabase format is in essence a MS Access format (.mdb) and as such the limitations can be overcome by users that have access to the MS Access software. Tabular Attribute Data: (a) MS Access format already stores the data in this format. Extracting data for distribution in this format will require the least effort by the staff. MS Access comes with the MS Office software which is very widespread software in use by many agencies. (b) Excel format is an alternative 19

20 option that widens the opportunities for compatibility with more software package. However, Metroplan should convert the data from Access to Excel. (c) Comma-delimited text file This format is the most compatible format; however its structure if fragile and easily corruptible if not handled carefully. In addition, the staff should convert the data from Access to comma-delimited text files. Recommendation: First choice recommended for crash data distribution from to partner agencies is ESRI s Personal Geodatabase (.mdb). This is a custom MS Access based format. It is well suited for crash data because the crash database organized in 9 tables is already stored and managed in Access. The personal geodatabase can store the crash point geocoded spatial data in the same Access database as the 10 th table. The processing time for staff would be minimal. All but one of the agencies in the area that use GIS are using ESRI s GIS software. Only City of Orlando uses MapInfo; however MapInfo can support MS Access files and can read ESRI s personal geodatabase directly. In the rare case of incompatibility with one or more partners, a second alternative would be to use MS Access database with the nine crash tables plus a 10 th table that contain the x,y coordinates of the crash point. This would be a standard Access format that is readable by ArcGIS, MapInfo, Geomedia or other GIS packages. Users of this format would have to convert the 10 th table to a spatial layer using the tools of GIS software of choice Distribution Methods Two options are recommended: if decides to open access of the crash data to anyone, the data should be provided on a web site for download. If distribution of the crash data would be limited to partner agencies the data can be made available for download from a web site or an ftp site with password protection. The size of the data is relatively small and it doesn t justify a distribution on a media such as CDROM or DVD although it is up to to make the decision to use its resources for burning and mailing media to agencies Data Documentations (metadata) It is strongly recommended that a well documented data dictionary is provided along with the data that is distributed or the data dictionary can be posted on the web site. DHSMV already provides the data dictionary for the crash attributes. can use this dictionary and expand it to include the description for the geocoded crash points Update Frequency Based on recommendation provided by the feasibility study referenced above, to fulfill the needs of law enforcement and engineering agencies the data sources must be current within 6 weeks. Considering that 20

21 needs some additional time to geocode and update the database, a two-month latency is a more realistic expectation for the data to be available for distribution. The frequency to actually put the new data out for distribution can be realistically determined by after testing the process for a few a months. should consider distributing the data at least quarterly (four times a year). However, testing may show that may be able to provide more frequent updates, potentially once a month Data Privacy Issues The DHSMV and PRIDE deliver the complete crash data including personal information such as names of drivers and pedestrians, vehicle owners, driver licenses etc. The data management tools provided to the staff are equipped with filters to optionally not include personal information in the crash database during the load process. If no personal information is needed for the analysis and reporting needs of partner agencies, can make the data available for download via a web site without limitation. Limited access may have to be established if some partner agencies require access to personal information Scanned Crash Reports There are two issues that present difficulties for the distribution of scanned reports. One, they contain all the personal information and two, their large file size is not suitable for web download. Scanned images can be distributed on DVD, however need to consider its available resources to accomplish this i.e. the staff time to burn the DVDs, cost of media, cost of mail, and distribution frequency. Before committing to providing this service should assess the partner agency needs to determine which agencies require access to scanned images to support their work related to safety improvements. To respect the data privacy requirements may chose to distribute the scanned reports to the partners provided that there is clearance about the personal information. also should consult the restrictions (if any) in the memorandum of understanding with DHSMV Conclusion can put out crash data for users that need them as a minimum 4 times a year although a higher frequency may be achieved if desired. The data is recommended to be distributed in a personal geodatabase format and in the same table structure as it is stored in the database. The data should be put out in a zipped file, by county, and named in a way that is self explanatory. An automated process that packages and places the data for ftp or website download is recommended for consideration. The download website should include a link to a well documented data dictionary. 21

22 4.2 Access to Analysis and Reporting Access to the data doesn t mean automatic access to analysis and reporting that can be done with the data. The users will need to make use of different software packages to use the downloaded data to fulfill their needs for reporting and analysis. The report Feasibility of Developing and Maintaining an Automated Vehicle Crash Data base in the Orlando Urban Area identifies the types of analysis and reporting that users would need. Using that recommended list, several illustrations of analysis and reporting that can be performed with the GIS crash data were developed as part of this project and they are provided in Appendix A Examples of Analysis and Reporting. Development of these kinds of analysis and reports requires knowledgeable staff and/or resources for paid services that partner agencies must have in order to convert the data into useful information. An alternative to the use of the local agency resources is for to provide the analysis and reporting services to the partners. Two scenarios can be considered: (a) Requests from partners are fulfilled by the staff. There are 25 law enforcement agencies and several city/county engineering and other agencies that are involved in the traffic safety area and have the need to work with traffic crashes. Although staff may be able to help these agencies with typical analysis and reports, considering the potential volume of these requests, this approach may not be sustainable and it is not recommended. (b) hosts a webbased crash analysis and reporting system that the partners can access to perform analysis and reporting themselves. The rest of this section discusses options and provides recommendations on the latter Web-Based Crash Data Analysis and Reporting System The concept for a web-based Crash Data Analysis and Reporting System maintained and hosted at the is presented in the figure 8 below: 22

23 PARTNER AGENCIES: can view, analyze, create reports & download data Law Enforcement Engineering Other FHP OPD INTERNET INTERNET Long and short paper forms METROPLAN ORLANDO Host System for Analysis & Reporting + data download GIS Crash Database (x y, data, images) METROPLAN ORLANDO Clean, Unify, Geocode Digital long forms and images Digital short forms and images HSMV Long and short paper forms PRIDE Digital long forms, all scanned images Figure 8 Concept for Web-Based Crash Data Analysis and Reporting System will continue to receive the crash data from PRIDE and DHSMV. Upon obtaining the data, the staff will use the current database and mapping tools to clean, unify and geocode the crash data. Next, the staff will upload the data into the database of the Crash Data Analysis and Reporting System. The Crash Data Analysis and Reporting System will provide reporting and analysis tools for the partner agencies. The partner agencies will be able to log in the system using only a web browser such as Internet Explorer or Firefox. Once logged in the users will be able to view, query, and analyze the data as well as create reports. In addition the system will continue to offer data 23

24 download for users that need to perform more advanced analysis using their software of choice. A web-based crash data analysis and reporting system is made of several components: hardware, software and data. The concept of System Architecture is typically used to explain the main components of such a system and the way they interact. A generic system architecture for a webbased GIS crash data system is presented in Figure 9 below: Figure 9 Web-based Crash Data System Architecture At the core of the system would reside one or more server computers that would store the database, the GIS engine and the crash analysis and reporting application. The database would be stored and managed using enterprise type Relational Database Management System (RDBMS) such as Oracle or MS SQL Server. In addition the database server would require a database component capable of storing spatial data such as GIS street centerlines, city boundaries etc. The GIS engine would require a server side GIS package such as ArcServer or Oracle Spatial. The application tools would be developed in java or in a.net environment and would be stored in the web server which is another component that is used to serve the information to the end-users via the internet. When a user wants to use a tool, e.g. see a map of high crash locations in a city or a county, the user s request will be received by the web server and analyzed by the application. The application will request the crash data from the database server and by using the GIS engine will calculate the high crash 24

25 locations and will produce a map which will be send back to the web server and from there displayed on the screen to the end-users. From the components described above, at present has access only to the MS SQL Server database system. To develop and host a system like the one described above would need to acquire additional hardware such as one or more servers and as well as software such as a server-side GIS system. In addition the application tools will need to be developed Configuration Recommendations Below are provided some suggested systems specifications for a webbased crash data system: Single Server Configuration For simplicity and economy, the crash system can be deployed on a single Windows server. The following is a recommended configuration. Recommended Hardware 3 GHz CPU 4 GB RAM 100 GB Disk Operating System Windows Server 2003 or greater Required Software IIS web server.net framework 2.0 ASP.Net AJAX extensions ArcIMS 9.2 SP6 or ArcGIS Server Oracle 10g ArcSDE 9.2 SP6 Dual Server Configuration For better performance and reliability, the web server and database components can be deployed on separate servers. Linux is preferred for the database server. The following is a recommended configuration. 25

26 Web server Recommended Hardware 2 GHz CPU 1-2 GB RAM 50 GB Disk Operating System Windows Server 2003 or greater Required Software IIS web server.net framework 2.0 ASP.Net AJAX extensions ArcIMS 9.2 SP6 or ArcServer Oracle instant client (10g) Database server Recommended Hardware 2 GHz CPU 4 GB RAM 50 GB Disk Operating System RHEL/CentOS/OEL Linux 4 or greater Required Software Oracle 10g ArcSDE 9.2 SP6 Note: If Oracle is unavailable, MS SQL Server can be considered as a replacement. The above configurations have been tested for a web-based crash data system developed by University of Florida using Oracle. 26

27 5. References: Feasibility of Developing and Maintaining an Automated Vehicle Crash Data base in the Orlando Urban Area,, December

28 6. Appendix A Examples of Analysis and Reporting The report Feasibility of Developing and Maintaining an Automated Vehicle Crash Data base in the Orlando Urban Area identifies the types of analysis and reporting that users would need. They are shown below ordered by importance based on the cumulative ranking of the surveyed agencies as described in the report above. (page 10). Only analytical or reporting needs are included: 1. Perform Ad-hoc Crash Data Analysis; Query Crash Data: This describes the ability of a user to isolate subsets of data based on user defined criteria, beyond what would be provided with standard reports. This might be used to monitor specific program areas or respond to categorical funding opportunities. 2. View GIS Crash Map of Individual Crash Locations This describes the ability of the system to display each individual crash event on the GIS map. 3. Generate Computerized Collision Diagram: This describes the ability to output computerized collision diagrams which illustrate crash types at an intersection or along a roadway segment 4. View GIS Aggregate Crash Maps: This describes the ability to aggregate crash data based on nearest intersection or roadway segment. Aggregate crash maps allows the user to see areas of high crash concentrations and can also show concentrations of specific pre-defined or user-defined crash types (ie. DUI, bike/pedestrian crashes, run-off-road crashes). 5. Generate Site-specific Reports, Charts, and Graphs: This describes the ability to generate reports, charts and graphs based on geographic locations including city limits, corridors, or intersections. 6. Generate System-wide or District-wide Reports, Charts, Graphs: This describes the ability to compare local data to larger data sets like county, district or state. Similar to DHSMV annual crash report. This purpose of this section is to provide some illustrations of such analysis and reports. These reports can be generated using MS Access, Excel, GIS or other tools. The examples below are not intended to demonstrate how these analysis or reports can be generated using any particular software but rather to provide an illustration of the analysis and reports listed above. The illustrations are provided in the following pages: 28

29 1. AD-HOC CRASH DATA ANALYSIS Query Input includes selection of a specific date range, a specific street and a DUI contributing cause 29

30 1. AD-HOC CRASH DATA ANALYSIS Query Results showing a pie chart of crashes that matched the query by cash type 30

31 1. AD-HOC CRASH DATA ANALYSIS Table showing a listing of crashes that matched the query 31

32 2. VIEW GIS CRASH MAP The maps show two different examples of displaying crash records; one for an area and another for a specific street or corridor 32

33 3. COMPUTERIZED COLLISION DIAGRAM The image below represent an automated intersection collision diagram. The red numbers beside each symbol represent the number of crashes for that crash type; The legend for the symbols is at the bottom. 33

34 4. VIEW GIS AGGREGATE CRASH MAP The map on the left shows all individual crashes; The map on the right shows crashes aggregated (summarized) by the nearest intersection; High concentrations can be seen 34

35 5. SITE SPECIFIC REPORTS, CHARTS GRAPHS The maps shows a crash type analysis for a specific intersection The bar chart shows the number of crashes by crash type for that intersection 35

36 5. SITE SPECIFIC REPORTS, CHARTS GRAPHS Distribution of crashes by contributing cause for a specific city; tabular reports and bar chart 36

37 6. SYSTEM / DISTRICT WIDE REPORT The maps show county wide report that identifies top 10 highest crash intersection and streets 37

38 6. SYSTEM / DISTRICT WIDE REPORT Tables and charts showing distribution of crashes by month for a given year for the entire system 38