Workflow improvement with FME in Skedsmo municipality Anders Hveem Malum (Geodata AS) and Turid Brox Nilsen (Skedsmo municipality) 16.05.2014 Skedsmo Kommune, Teknisk sektor 1
Skedsmo municipality Situated between Oslo and Gardermoen airport Lillestrøm is the administrative centre of a growing municipality 50 000 inhabitants (2013) Area 77 km 2 25 000 jobs Lillestrøm Centre of Expertise one of Norway s largest research- and technology environments Skedsmo is host of Norway Trade Fairs 28.01.2013 Skedsmo Kommune, Teknisk sektor 2
GIS in Skedsmo municipality Geodata department with 23 employees Surveying Updating the cadastre Property taxation Manage other public fees One team of six people adminster databases and makes maps available ArcGIS FME Server and Desktop GeoSOSI Professional GDMMapper 16.05.2014 Skedsmo Kommune, Teknisk sektor, 3
Geodata AS About Geodata Established 1988 Distributor of Esri and Safe Software in Norway 115 employees Offices in Oslo, Stavanger, Hønefoss and Steinkjer What do we offer? ArcGIS software from Esri FME software from Safe Customized solutions Consulting and assessment services Map products and GIS - data Online hosting and GIS Services in the cloud Courses and support 16.05.2014 Skedsmo Kommune, Teknisk sektor, Geodata AS 4
FME use case: Zoning plans from A to Z The workflow: Initiator (member of the public / private firm / government agency) constructs and sends us the zoning plan as a SOSI-file Skedsmo converts the SOSI-file to a work-geodatabase (GDB) using FME and GeoSosi Professional (SOSI Plugin from Geodata) Quality control in ArcMap Is there already data where the new plan is situated? Old data is extracted into a backup geodatabase (GDB) Data from the GDB is loaded into the full database (SDE) using an ETL Tool in ArcMap Users access FME projects as jobs on FME Server or as ETL Tools in ArcMap. Software used: FME Workbench 2014, extended to ArcGIS GeoSOSI Professional FME plugin from Geodata ArcMAP 10.2.1 with Python 2.7 16.05.2014 Skedsmo Kommune, Teknisk sektor Geodata AS 5
FME use case: Zoning plans from A to Z Most of the tools for zoning plans are FME ETL Tools, combined with ArcGIS ModelBuilder and some Python scripting: Toolbox Python scripts FME ETL tools Model 16.05.2014 Skedsmo Kommune, Teknisk sektor Geodata AS 6
FME use case: Zoning plans from A to Z Example 1: Loading proposed plan from SOSI-file to Geodatabase, and create a layer file for the plan. Tool input: 16.05.2014 Skedsmo Kommune, Teknisk sektor, Geodata AS 7
FME use case: Zoning plans from A to Z Example 1 what happens when the user pushes the OK button? The FME ETL tool is executed: A new file geodatabase is created using a xml schema template for proposed plans FME reads input SOSI file Data from the SOSI file is imported, and mapped to the correct feature classes and subtypes in the newly created file geodatabase. Python script 1 starts This script loops through the file geodatabase and deletes any feature classes in the template schema that are empty after the SOSI - import. Python script 2 starts This script uses standard presentation from GeoAreal (geodata product) to symbolize the newly created file geodatabase. All feature classes from the new file geodatabase is added to the map document with correct representation and symbols Zoning plan topology is also validated and symbolized. 16.05.2014 Skedsmo Kommune, Teknisk sektor, Geodata AS 8
FME use case: Zoning plans from A to Z The log from FME and from the python scripts is written to the GP tool results: FME log Python log 16.05.2014 Skedsmo Kommune, Teknisk sektor, Geodata AS 9
FME use case: Zoning plans from A to Z And it is also available from the Geoprocessing results dialog in ArcMAP afterwards: 16.05.2014 Skedsmo Kommune, Teknisk sektor, Geodata AS 10
FME use case: Zoning plans from A to Z Sample output from the tool: 16.05.2014 Skedsmo Kommune, Teknisk sektor, Geodata AS 11
FME use case: Zoning plans from A to Z Example 2: Extracting «expired» data from the SDE database: To be deleted from SDE 16.05.2014 Skedsmo Kommune, Teknisk sektor, Geodata AS 12
FME use case: Zoning plans from A to Z Example 2: Extracting «expired» data from the SDE database: 16.05.2014 Skedsmo Kommune, Teknisk sektor, Geodata AS 13
FME use case: Zoning plans from A to Z Example 2: Extracting «expired» data from the SDE database: Input SDE database Outline of new plan File GDB with clipped data Delete/update on SDE database Clipper 16.05.2014 Skedsmo Kommune, Teknisk sektor, Geodata AS 14
Water and sewerage installation data Skedsmo municipality maintains their water and sewerage installations data in an Oracle database using the program Gemini from the Norwegian company Powel. FME Workbench is used to convert those data into an ArcGIS Geodatabase. The FME Project also detects logical errors in the original data, and writes error logs and feature classes containing the objects with errors. Code errors (deviations from the Norwegian SOSI - standard) Network errors A scheduled Python script runs this conversion every week, and sends an e-mail containing extracts of the logs to system administrator. 16.05.2014 Skedsmo Kommune, Teknisk sektor, Geodata AS 15
Water and sewerage installation data - example Topology/network errors Code errors 16.05.2014 Skedsmo Kommune, Teknisk sektor, Geodata AS 16
Water and sewerage installation data - log from scheduled routine FME Errors FME Warnings FME Statistics 16.05.2014 Skedsmo Kommune, Teknisk sektor, Geodata AS 17
Use case: «Sak i kart» A map layer that shows cases we are working on from our electronic filing system Ie. Building applications, new addresses, new/changed parcels FME extracts data from our electronic filing system using the cadastral/property unit number (gnr./bnr.) to georeference the points A python script in FME spreads the points from one parcel apart so they don t overlap Sceduled job on FME Server updates the points nightly. Data is presented in a map inside our firewall, there are links directly from the points to the file in the electronic filing system Very useful to get an overview of everything that is happening on a parcel 16.05.2014 Skedsmo Kommune, Teknisk sektor, Geodata AS 18
16.05.2014 Skedsmo Kommune, Teknisk sektor, Geodata AS 19
Use case: work list for changes to the cadastre Norwegian: «Arbeidsliste for matrikkelførere» Changes in our cadastre as a result of building permits is input into software from the Norwegian Mapping Authority. Challenge: How do we find out which building permits/changes to a building needs to be input into the cadastre? FME goes through our electronic filing system, looks at case statuses and writes this to an Excel spreadsheet The spreadsheet is updated nightly using a scheduled job on FME Server The person updating the cadastre edits the spreadsheet to show when a case has been updated in the castre 16.05.2014 Skedsmo Kommune, Teknisk sektor, Geodata AS 20
Use case: data compilation and adaptation in a property tax project (1/3) We need to appraise (taksere) 15 000 parcels in Skedsmo A valuation board (takstnemnd) approves all parcel values We make an estimate of a parcel s value based on data in the cadastre (building size etc.) One important factor in the estimate is the «neighborhood factor» in an area with normal parcel prices this will be 1 in an area with relatively expensive parcels higher (for instance 1,3) in an area with relatively cheap parcels lower (for instance 0,8) The «neighborhood factor» is calculated based on actual sales figures The «neighborhood factor» is different in parcels with different building types (for instance detached houses vs. apartment blocks) Challenge: assign a «neighborhood factor» to records in the property tax database Building blocks: property tax database, cadastre data, polygons containing the neighborhood factor 16.05.2014 Skedsmo Kommune, Teknisk sektor, Geodata AS 21
Use case: property tax project (2/3) Buildings in the cadastre have a building type Georeferenced tax record Tax records needs to have the same information Solution: use FME to assign building type to the parcel, then use the geometry of the parcel to assign a building type value to the tax record Then use the building type of the tax record to assign it a «neighborhood factor» Parcel Point from the cadaster with Building type Ie. detached house 16.05.2014 Skedsmo Kommune, Teknisk sektor, Geodata AS 22
Use case: property tax project (3/3) Obligatory illustration showing a confusing FME project 16.05.2014 Skedsmo Kommune, Teknisk sektor, Geodata AS 23
What we gain by using FME Programming made visual Ensures consistency and lessens the possibillity of errors when loading/ extracting data Saves employees in Skedsmo lots of time by automating tasks FME in combination with the ArcGIS platform is a very powerful and efficient way of improving workflows for every day GIS tasks. Get in touch: Anders Hveem Malum (anders.hveemmalum@geodata.no) / Turid Brox Nilsen turnils@skedsmo.kommune.no 16.05.2014 Skedsmo Kommune, Teknisk sektor, Geodata AS 24