Enterprise Imagery Management at NYSDOT 2011 GIS-T Symposium March 30, 2011
Use of imagery is pervasive at any DOT ArcGIS Desktop Users GIS Web Applications Microstation CAD users
The NYS Digital Orthoimagery Program (NYSDOP) Statewide coverage 4-band orthoimagery is capable of both natural color and color infrared display 0.5 2 foot pixel
Raster geospatial data used by NYSDOT NYSDOT Photogrammetric Products (Project Based) Design scale orthoimagery mosaics (1 =40ft) are constructed for project areas.
Raster geospatial data used by NYSDOT Elevation datasets including county and project based LIDAR
Project 3805.19 Terrestrial LiDAR (Light Detection and Ranging) data. Purple crosses delineate surveyed control points used to tie LiDAR point clouds into mapping. Data set consists of 108 million points. Different colors represent different reflectance intensities Reflectance intensity is a result of the distance the laser beam traveled, angle of laser beam incidence and the type of material that is struck.
Region 205675 Utica. A combination of survey DTM, terrestrial Lidar and photogrammetric mapping
PIN 935783 Photogrammetry Airborne Lidar of approximately 6 million points. Terrain model of Lidar data (upper left). The 2006 Orthoimagery draped over the terrain model (lower left). Photography layout and survey control points (lower right).
Photo and Mapping Index Online index identifies orthoimagery and photos available for NYSDOT users.
Legacy imagery management Image catalog (2001-2006) The large numbers of image tiles Mediocre performance in ArcGIS Individual SID/JP2 tiles can be imported in the Bentley CAD environment There are 2497 tiles in Westchester County. Over 250,000 tiles cover New York State.
File geodatabase mosaic (2006-2010) Much better performance in ArcGIS than the image catalog. Building the mosaic from the tiles is a time consuming process. Useless to the community of CAD users who can not consume an Esri geodatabase. Legacy imagery management The geodatabase mosaics are very large - Westchester County is 32 GB
What s wrong with the current methods? Performance Display speed of orthoimagery is completely unacceptable in remote offices with limited connectivity to the file system. A large footprint on disk FIle geodatabase compression is less efficient than the original tiles. File geodatabases used up to 6 X more disk space. 10-70GB per county. Way too many footprints File based versions of orthoimagery are being redundantly stored in multiple locations: Main Office, Regional Offices, Residencies, external hard drives, etc. Esri specific data formats are only useful to Esri clients Bentley CAD users do not have easy access to NYS Orthoimagery. Publishing a new season s orthoimagery is a manual, time consuming task. Assembling and publishing image catalogs or file geodatabases requires significant resources. Field devices are not connected to the Department s network and therefore not able to access imagery.
Enterprise Image Management Goals Service oriented architecture High performance Easy to maintain Fewer copies of imagery data Available to GIS and CAD clients Imagery and LIDAR data available for analysis.
Results of ArcGIS Server Image Extension testing (Summer 2010) Service-oriented architecture ArcGIS Server Image Extension provides Esri services or WMS. Web services require far less network bandwidth than file based imagery. High performance Using development tier hardware, testing showed high performance regardless of location and network bandwidth. Easy to maintain ArcGIS Server Image Extension may be set up to use NYS imagery tiles in the delivered format with no post processing. Imagery may be published within hours of receiving. Fewer copies of imagery data Imagery is available to all locations on the NYSDOT network and the internet from a single source. Eliminate the current duplication of large imagery datasets on regional file systems. Available to GIS and CAD clients ArcGIS Server Image Extension services successfully tested in Esri and Microstation XM and 8i Imagery and LIDAR data available for analysis. ArcGIS Desktop can complete any type of analysis using imagery or LIDAR data.
Esri System Architecture Review - Jan 2011 Our review estimated the total concurrent usage of 200 users. User Community Editors (ArcGIS Desktop) 100/10 Viewers (ArcView) 1400/90 Viewers (Microstation CAD) 4000/100 Web Services Mobile Total User/Concurrent Users 5000 hits per hour 1000 hits per hour New System requirements for Image Server at NYSDOT: Application Servers: Five (5) Physical ArcGIS Servers (up to 30 2-core Virtual Partitions/6 per server) Xeon X5680 12 core (3.33GHz), 48 GB RAM per server Storage: Additional 10 TB SAN storage: 10K speed, 2 TB disks Total investment: approx $102,000
Next steps We are leveraging federally funded enterprise projects to help fund the procurement new hardware. Moving the ArcGIS Server / Image Extension to version 10. Complete development of Image Extension processes for NYS image and LIDAR data. Deploy services to separate user communities. 1. ArcGIS Server applications 2. ArcGIS Desktop clients 3. Microstation 8i client using WMS Carefully monitor concurrent use and benchmark performance as we deploy to each new community.
Conclusions Imagery is a critical data source for many aspects of the DOT mission but maintaining and serving large imagery datasets has traditionally been a time consuming challenge. An enterprise, service oriented imagery management tool can simplify the process of publishing imagery and provide a wider audience easy, high performance access to valuable imagery and LIDAR datasets.
Demonstrated benefits Image Server Web services based solution that will be available for use in internal and external GIS web services. Will provide imagery to the entire enterprise. ESRI GIS desktops and CAD (Microstation) clients. Orthoimagery is maintained and served from a single location. This simplifies maintenance and prevents very large imagery datasets from being replicated on file systems in the Main Office, Regions, and Residencies. Increased Performance; increased performance in at all NYSDOT locations. Testing at low bandwidth sites such as Residencies shows up to 400% faster performance. More efficient use of network bandwidth using Image Server overviews. Single source of imagery distributed by web service statewide. Eliminate the current duplication of large imagery datasets on regional file systems. Smaller footprint on the file system compared to File Geodatabase image mosaics. Increase performance in delivering basemap rasters such as quadrangle maps. Enterprise access to photogrammetry and design project imagery products. Image Server will greatly enhance CAD workflows by allowing direct access to NYS orthoimagery. This prevents CAD users from download imagery on a per project basis and storing copies of this imagery in the Projectwise environment. Image Server will provide image services to users with much less effort than the alternatives. Image Server uses orthoimagery in the native format delivered by NYS OCSCIC, saving significant ITD GIS time when publishing new coverage. Image Server services are capable of providing natural color or color infrared to the user from the new NYS 4-band imagery. Image Server licenses are server based and included in the NYSDOT ESRI Enterprise License Agreement. There is no client side licensing necessary (unlike other products)
Thank you! Contact the following contributors for more information: Kevin Hunt, GISP GIS Manager, NYSDOT GIS kevin.hunt@dot.state.ny.us Robert Rizzo, Mapping Technologist, NYSDOT GIS robert.rizzo@dot.state.ny.us David Hess, Mapping Technologist, NYSDOT GIS david.hess@dot.state.ny.us Ron Frederiks, CP ASPRS, Photogrammetry Manager, NYSDOT Design rfrederiks@dot.state.ny.us Eric Floss, Esri Transportation Practice efloss@esri.com