SECTION 1. INTRODUCTION...

Transcription

1 USDA Carbon Data Visualization Tool Configuration Manual Administration Guide Version 1.0 August 2012 Critigen and The catalyst to modernize critical infrastructure are trademarks of Critigen. All other trademarks used herein are the property of their respective owners. Copyright 2010 Critigen. All rights reserved.

2 Contents SECTION 1. INTRODUCTION BACKGROUND PURPOSE AUDIENCE DEFINITIONS, ACRONYMS, AND ABBREVIATIONS REFERENCES AND RELATED DOCUMENTS... 5 SECTION 2. SOLUTION OVERVIEW SOFTWARE APPLICATION Main Application (i.e., ASP.NET) Major Components Application Structure Security Software Components/Versions Mapping Application (i.e., ArcGIS Server, Rest API, Flex) Header Controller Menu Map Navigation Toolbar Charting Identify Legend Layers Data Availability/Selection Find Swipe Time Series User Preferences Overview Map Software Components/Versions NET Web Service Public Web Methods (Called from Flex) Private Functions (Called from within.net web service) SQL SERVER DATABASES Spatial Database Model Scenario Data Sets Carbon Field Mapping Carbon-Related Data Sets Reference Data Sets Application Dependent Data Sets Non-Spatial Database Business Tables Security Tables SQL Express Directory Structure & Security Dynamic Map Services Basic Map Services Time Series Map Services Carbon Composite Map Service Cached Map Services Third Party Map Services SECTION 3. SOLUTION ADMINISTRATION SOFTWARE APPLICATION MANAGEMENT Managing Model Information P a g e ii

3 Update Model Label, Name, and Description Update Model Documentation Add Model Documentation Delete Model Documentation Update Model Scenarios Add Model Scenarios Delete Model Scenarios Update Model General Information Update Model Input Update Model Output MANAGING MAP LAYER INFORMATION MANAGING USERS AND ROLES DATABASE MANAGEMENT Spatial Data Management New Map Layers ArcGIS Server Map Service Creation ArcGIS Server Cache Creation Updated Map Layer Availability Feature Class Time Series Map Layers Non-spatial Data Management Add Carbon Model Scenario Data Set Add Carbon Data Set Add Reference Data Set Enable/Disable Charting Enable Identify Enable Carbon Composite Enable Find Enable Time Series...56 SECTION 4. SYSTEM DEPLOYMENT AND CONFIGURATION SYSTEM DEPLOYMENT Databases Map Services Web Service Main Application Mapping Application APPLICATION CONFIGURATIONS Flex Configuration File Web Service Configuration File Database Updates File System Access Legend Images Model Documents Error Log File Server Configurations IIS Configuration HTTPS Configuration Reference SECTION 5. ASSISTANCE AND PROBLEM REPORTING P a g e iii

4 Figures Figure 2.1: Header Controller Menu Figure 2.2: Map Navigation Toolbar Figure 2.3: Pie Chart Widget Figure 2.4: Bar Chart Widget Figure 2.5: Identify Widget Figure 2.6: Carbon Identify Widget Figure 2.7: Legend Widget Figure 2.8: More Info Widget Figure 2.9: Layer List Widget Figure 2.10: Model Data Availability Widget Figure 2.11: Model Data Selection Widget Figure 2.12: Find Widget Figure 2.13: Swipe Widget Figure 2.14: Time Series Widget Figure 2.15: User Preferences Widget Figure 2.16: Overview Map Widget Figure 2.17: Carbon-related Data Sets Figure 2.18: Mapping Application High-Level Database Structure Figure 2.19: Security High-Level Database Structure Figure 2.20: Manage Services Tab Figure 2.21: Manage Folder Options Figure 2.22: Permissions Mapping Figure 2.22: CDVT-P Map Service Folder Structure in ArcGIS Server Manager Figure 2.23: CDVT-P Map Service Folder Structure in ArcCatalog Figure 3.1: Model Information Link Figure 3.2: Model Information Summary Page Figure 3.3: Model Information Page Figure 3.4: Update Model Information Page Figure 3.5: Edit Model Documentation Figure 3.6: Update Model Documentation Page Figure 3.7: Add Model Documentation Figure 3.7: Add Model Documentation Page Figure 3.8: Delete Model Documentation Figure 3.1: Map Scale Settings Figure 3.2: Publish to ArcGIS Server Figure 3.3: Publish Map Service Dialog Figure 3.4: Create Map Service Cache Figure 3.5: Initial Levels of Detail To Create Figure 3.6: Manage Map Server Cache Tiles Figure 3.7: Map Layer Time Properties Figure 3.8: Time Tools and Slider P a g e iv

5 Section 1. Introduction 1.0 Background The USDA Carbon Data Visualization Tool Prototype (CDVT-P) organizes and make readily accessible carbon-related information owned or used by USDA via a single web site. The application information includes Geographic Information System (GIS) data layers, model information; model results, related information, and tools to improve access to and display of the information. 1.1 Purpose This manual describes how to set up and administer the CDVT-P product. 1.2 Audience This document is intended for Data Center managers, USDA system managers, and application developers. 1.3 Definitions, Acronyms, and Abbreviations Acronym/Name/ Abbreviation ASP CDVT-P FORCARB2 GIS IIS MXD SMTP UI USDA WCF XML Definition Active Server Pages Carbon Data Visualization Tool - Prototype Forest Carbon Budget Estimator Geographic Information System Internet Information Services Map Document Simple Mail Transfer Protocol User Interface United States Department of Agriculture Windows Communication Foundation Extensible Markup Language 1.4 References and Related Documents USDA Carbon Data Visualization Tool Software Users Manual v1.0 technical communication document intended to give assistance to USDA CDVT-P users. Esri s ArcGIS Server Installation Guide ml#/windows_server_2008_r2_iis_requirements/ n000000/ Esri s ArcGIS Server Microsoft.NET Framework Installation Guide ml Page 5 of 64

6 Esri s ArcGIS Server SSL Setup Guide p_ssl/ pv000000/ Esri s ArcGIS Server Administration Guide Esri s ArcGIS Server Security Guide ough_implementing_security_with_users_and_roles_in_sql_server/ pr000000/ Esri s ArcGIS Server Get Token Guide 000q Page 6 of 64

7 Section 2. Solution Overview 2.1 Software Application The CVDT-P application consists of: 1. Main Application An ASP.NET 4.0 web application containing the login and welcome, the embedded mapping application, model and map layer information, and administrative tools. 2. Mapping Application A customized ArcGIS Server Flex Viewer application. 3..NET Web Service A WCF web service providing numerous web methods for retrieving and saving non-spatial CDVT-P data Main Application (i.e., ASP.NET) The main application was written in ASP.NET 4.0 and consists of the user interface modules and navigation framework. Persistent data is stored by the.net application in a SQL Server 2008 database Major Components The major components of the ASP.NET application are: 1. Login / Account Management Users can register, log in, and manage their account. 2. Welcome Page Users can view welcome information, contacts, and pertinent links 3. Carbon Visualization Tool Users can access the Carbon Visualization Tool mapping module. (Note: The Carbon Visualization Tool mapping module is a Flex application/flash file that is embedded in the ASP.NET application. Please see Section for technical details on the Carbon Visualization Tool.) 4. Model Information View/Management Users can view and edit Model metadata. 5. Map Layer Information View/Management Users can view and edit Map Layer metadata. 6. Administrative Tools Privileged users can manage user accounts, role privileges, Welcome content, links, and lookup tables. Each of these components has a set of user interface pages, one or more business objects, and a set of SQL Server stored procedures that support it Application Structure This section outlines where the code and files for each of the major ASP.NET application components can be found. Page 7 of 64

8 1. The interface pages are located in the following application directories: <Root> Account Map ModelInfo MapLayerInfo Admin 2. The primary business object classes are in the Components application directory: User.vb Model.vb MapLayer.vb Link.vb Lookup.vb Message.vb 3. The source files for the SQL Server stored procedures that support the business object classes are in the Database\StoredProcedures directory in the code archive. 4. Privileged users can upload Model Documentation files to the website. These are stored in the Documents application directory in the following subdirectories: Documents\ModelDocumentation Documents\ModelInputSamples Documents\ModelOutputSamples 5. Privileged users can also upload Legend Image files to the website. These are stored in the Map\Widgets\ModelDataViewer\Images\Legend\ application directory. 6. The application writes unexpected/unhandled errors to an ErrorLog.txt file in the root of the virtual directory. 7. The application sends s to users and application administrators. The HTML templates are located in the Templates application directory. 8. The application uses a third party open source tool called TinyMCE for content management functionality. The TinyMCE files are located in the TinyMCE application directory Security The CDVT-P ASP.NET application supports three user roles: Public User, Manager, and Administrator. The user s role determines the tools to which the user has access and the layers to which the user the access. User role privileges are stored in the database and can be changed by privileged users in the Manage Roles module.the CDVT-P application uses the ASP.NET Membership Provider for user account authentication, storage, and management. It also uses ASP.NET Forms Authentication for secure and privileged page access. The CDVT-P deployment environment uses an SSL certificate to encrypt data transmitted to and from the application Software Components/Versions The CDVT-P ASP.NET application uses or requires the following software components: Page 8 of 64

9 1. Microsoft Internet Information Server (IIS) 2. Microsoft.NET Framework jquery TinyMCE Internet Explorer 7/8/9, Firefox 3.6/5, Chrome 12, or Safari Adobe Flash Player 9 or higher 7. Microsoft SQL Server Simple Mail Transfer Protocol (SMTP) Mapping Application (i.e., ArcGIS Server, Rest API, Flex) The tools and major components are initialized and controlled by editing Extensible Markup Language (XML) configuration file. The USDA Flex viewer application has a configuration file named config.xml. It controls the appearance, functionality, and data content in the Viewer application. It is located in the main flex viewer installation folder. The individual tool, known as widgets, also has a configuration file which control appearance and functionality. The contents of config.xml can be divided into the following main sections: General properties, for example application title, style colors, fonts Application user interface (UI), for example the header control and toolbar Map content, for example base maps Widget container, for example Find and Identify Services and application settings custom web services used to communicate with databases General properties: <title>carbon Model Data Viewer</title> <subtitle>carbon Model Data Viewer</subtitle> <style> <colors>0x000000,0xffffff,0xcccccc,0xffffff,0x000000</colors> <alpha>0.8</alpha> </style> Map content: <layer label="aerial" type="tiled" visible="true" url=" [ArcGIS REST URL] /World_Imagery/MapServer" <layer label="counties" type="dynamic" visible="true" visiblelayers="0" alpha="1" url="[arcgis REST URL]/common/PublicPortalData/MapServer" <layer label="data Availability" type="dynamic" visible="false" alpha="1" url="[arcgis REST URL]/common/PublicPortalData/MapServer" Widgets: Page 9 of 64

10 <widgetcontainer layout="float"> <widget label="layers" icon="widgets/modeldataviewer/images/application_view_list.png" config="widgets/modeldataviewer/toc/maploaderwidget.xml" url="widgets/modeldataviewer/toc/maploaderwidget.swf" <widget label="identify" left="330" top="80" icon="widgets/modeldataviewer/images/information.png" config="widgets/modeldataviewer/identify/identifywidget.xml" url="widgets/modeldataviewer/identify/identifywidget.swf" Services and application settings: <modeldatawebservice>[web Service URL]/service.svc?wsdl <queryavailability>[arcgis REST URL]/PublicPortalData/MapServer/1 <querycounty>[arcgis REST URL]/PublicPortalData/MapServer <iconimagedirectory>widgets/modeldataviewer/images/ <helpdocument>[web Application URL]/Documents/Help/USDA_Software_Users_Manual_SUM_v0.1.pdf <availabilityfield>[arcgis REST URL]/PublicPortalData/MapServer/0 <availabilitycounty>[arcgis REST URL]/PublicPortalData/MapServer/0 <availabilitystate>[arcgis REST URL]/PublicPortalData/MapServer/3 <availabilityusfs>[arcgis REST URL]/PublicPortalData/MapServer/ Header Controller Menu The header in the USDA application is actually a widget; it has been modified from the original Esri widget for the USDA launch other widgets or links. HeaderController config.xml entry: <widget left="0" top="0" config="widgets/headercontroller/headercontrollerwidget.xml" url="widgets/headercontroller/headercontrollerwidget.swf" Map Navigation Toolbar Figure 2.1: Header Controller Menu This is the standard navigation widget that has been customized with additional buttons to launch map location based tools. This widget can be minimized but not resized. Toolbar config.xml entry: <widget label="toolbar" left="25" top="30" config="widgets/modeldataviewer/toolbar/modeldataviewertoolbar.xml" url="widgets/modeldataviewer/toolbar/modeldataviewertoolbar.swf" Page 10 of 64

11 Minimize Bar Chart Pie chart Carbon identity Identify Model Selection Pan Zoom In to selection Zoom In Zoom Slider Zoom Out Zoom Out to Selection Full Extent Previous Extent Next Extent Clear Map Figure 2.2: Map Navigation Toolbar Charting The charting widgets are custom but follow the standard Esri template for Flex widgets. The charting tools are activated by the toolbar and require a coordinate (by clicking on the map); the coordinate is used to query data available at that point. The data is then returned to an infowindow which is displayed where the user clicked. An infowindow is an Esri map pop up window. Data presented in the info window is grouped within tabs labeled by data aggregates Regional, State, County and Sub County data. All the carbon data is defined by one of these aggregate so that like data can be displayed together for the user. Charting config.xml entries: <widget label="model Scenario Bar Charts" left="330" top="80" icon="widgets/modeldataviewer/images/chart_bar.png" config="widgets/modeldataviewer/barchart/barchartwidget.xml" url="widgets/modeldataviewer/barchart/barchartwidget.swf" <widget label="model Scenario Pie Charts" left="330" top="80" icon="widgets/modeldataviewer/images/chart_pie.png" config="widgets/modeldataviewer/piechart/piechartwidget.xml" url="widgets/modeldataviewer/piechart/piechartwidget.swf" Charting config file has many settings but really the only ones that should be edited are listed below. <simplefillsymbol color="0x448ccb" alpha="0.4" style="solid"> <outline color="0x448ccb" alpha="0.8" width="2"/> <simplemarkersymbol style="circle" size="12" color="0x448ccb" alpha="0.8" xoffset="0" yoffset="0" angle="0"> <outline style="solid" color="0xffffff" alpha="0.8" width="1"/> Page 11 of 64

12 <picturemarkersymbol url="assets/images/i_info.png" height="30" width="30" xoffset="0" yoffset="0" /> <simplelinesymbol color="0x448ccb" alpha="0.8" width="2"/> Figure 2.3: Pie Chart Widget Identify Figure 2.4: Bar Chart Widget The Identify and Carbon Identify are both widgets but Carbon Identify is more like the Charting widgets, whereas the Identify is a standard GIS tool. Carbon Identify displays data at aggregate levels like the Charting but does not use the Info window, its data is displayed using the flex popup as independent of the map. The Carbon Identify was created to display specific data about the carbon models where values are totaled by aggregate levels. The Identify tool is simply for displaying a record in a feature class (point, polygon, polyline, and raster) and this tool is primarily for non-carbon data. Identify config.xml entry: <widget label="identify" left="330" top="80" icon="widgets/modeldataviewer/images/information.png" config="widgets/modeldataviewer/identify/identifywidget.xml" url="widgets/modeldataviewer/identify/identifywidget.swf" Carbon Identify config.xml entry: <widget label="carbon Identify" icon="widgets/modeldataviewer/images/legend.png" config="widgets/modeldataviewer/carbonidentify/carbonidentifywidget.xml" Page 12 of 64

13 url="widgets/modeldataviewer/carbonidentify/carbonidentifywidget.swf" Identify & Carbon Identify config file settings: <simplefillsymbol color="0x448ccb" alpha="0.4" style="solid"> <outline color="0x448ccb" alpha="0.8" width="2"/> <simplemarkersymbol style="circle" size="12" color="0x448ccb" alpha="0.8" xoffset="0" yoffset="0" angle="0"> <outline style="solid" color="0xffffff" alpha="0.8" width="1"/> <picturemarkersymbol url="assets/images/i_info.png" height="30" width="30" xoffset="0" yoffset="0" /> <simplelinesymbol color="0x448ccb" alpha="0.8" width="2"/> Figure 2.5: Identify Widget Figure 2.6: Carbon Identify Widget Legend The Legend is a database driven tool that relates a legend image to a layer within the map. The standalone widget displays legend images that have been loaded into the map. The legend can also be accessed through the more info window which can be accessed through out the site where displaying layer information. The legend widget further defines the data types with different icons for Model Scenarios, Data, Reference Data. Model Scenarios Carbon model output data sets such as FORCARB2 or Century Page 13 of 64

14 Data - Carbon-related data sets such as SSURGO or precipitation Reference Data Referencial (base map) data sets such as aerials and county layers Legend config.xml entry: <widget label="legend" icon="widgets/modeldataviewer/images/legend.png" config="widgets/modeldataviewer/legend/legendwidget.xml" url="widgets/modeldataviewer/legend/legendwidget.swf" Legend widget config settings: <usebasemaps>true</usebasemaps> <useoperationallayers>true</useoperationallayers> <respectcurrentmapscale>true</respectcurrentmapscale> Figure 2.7: Legend Widget Figure 2.8: More Info Widget Layers Page 14 of 64

15 The layer widget contains all the data that can be added to the map and is ordered by data types. The maps are added in the order that they are chosen, the last one being on top. The layer widget can change the opacity in order to control the transparency of the layer. Layer widget config.xml entry: <widget label="layers" icon="widgets/modeldataviewer/images/application_view_list.png" config="widgets/modeldataviewer/toc/maploaderwidget.xml" url="widgets/modeldataviewer/toc/maploaderwidget.swf" Figure 2.9: Layer List Widget Data Availability/Selection The data availability and selection tool are carbon model specific widgets. The model data availability is made up of two tools that show the availability of the layers as a graphic on the map. The Data Availability tool located in the toolbar is used to highlight all the models in the Availability Viewer from a user defined geographic extent. The tool uses a special polygon (Called Availability) feature class that is a foot print of the models extents. The user defined extents are used to query the feature class where overlap occurs. The layers are highlighted blue in the Availability viewer and all the graphic representations are added to the map. The clear map tool can be used to remove all graphics and highlighting. Required config.xml entries: <availabilityfield>[arcgis REST URL]/PublicPortalData/MapServer/0</availabilityfield> <availabilitycounty>[arcgis REST URL]/PublicPortalData/MapServer/0</availabilitycounty> <availabilitystate>[arcgis REST URL]/PublicPortalData/MapServer/3</availabilitystate> <availabilityusfs>[arcgis REST URL]/PublicPortalData/MapServer/2</availabilityusfs> <widget label="select Model" icon="widgets/modeldataviewer/images/selectionmodel.png" config="widgets/modeldataviewer/selectmodel/selectmodelwidget.xml" url="widgets/modeldataviewer/selectmodel/selectmodelwidget.swf" Page 15 of 64

16 Figure 2.10: Model Data Availability Widget Figure 2.11: Model Data Selection Widget Find The Find widget allows users to quickly locate and zoom to an area of interest (e.g., county, state, USFS region, or Federal Lands). Additional map layers can be added to the Find tool (i.e., the Map Layer dropdown) by setting the DataSet table s IsSearchable field to true and including the appropriate code in the Find widget. Page 16 of 64

17 Required config.xml entries: <widget label="find" left="330" top="80" icon="widgets/modeldataviewer/images/earth_find.png" config="widgets/modeldataviewer/find/modeldataviewerfindwidget.xml" url="widgets/modeldataviewer/find/modeldataviewerfindwidget.swf" Swipe Figure 2.12: Find Widget The Swipe widget allows users to compare two different map layers through an overlay. All map layers available in the Layers widget (i.e., all data that can be added to a map) are loaded into Swipe tool map layer dropdown lists Time Series Figure 2.13: Swipe Widget The Time Series widget allows users to view a time lapse of a selected map layer. Only a subset of map layers in CDVT-P are time-aware. The time-aware map layers are managed in the TimeSeries table. The table contains information on the start and end dates for the time sequence as well as the time internval (e.g., month or year). See Section for more information on prepping a map layer for use in the Time Series tool User Preferences Figure 2.14: Time Series Widget The User Prefences widget allows users to view and control settings for the display unit as well as data sets and fields to be included in the charting, identify, and carbon composite layers. Default selections for the Carbon Visualization group are captured in the DataSet and modelcarbonfieldmapping tables. See Section Models, Scenarios, and Carbon Pools for more information. Page 17 of 64

18 Figure 2.15: User Preferences Widget Overview Map The Overview Map widget allows a user to view his/her current location or quickly navigate to an area of interest in the full extent of the map. The Overview Map contains only the counties and aerial base maps for reference purposes Software Components/Versions Figure 2.16: Overview Map Widget Requirements for the CDVT-P are: Web Server o Windows Server 2003 or above o IIS 7.01 or above o.net Framework 4 o ArcGIS Server 10 SP 2 Database Server o Windows Server 2003 or above o SQL Server 2005 o SQL Server Express o Esri ArcSDE Page 18 of 64

19 Client o Esri ArcGIS Desktop (ArcCatalog and ArcMap) NET Web Service The.NET web service provides easy retrieval and storing of non-spatial CDVT-P. Many of the methods within the.net service are public, being called externally from the main and mapping applications. A handful of the methods are private and can only be accessed from the methods within the.net service. The following sections provide high-level descriptions of the numerous methods available in the.net web service Public Web Methods (Called from Flex) GenerateUserCarbonCompositeTable() Executed from the User Preferences widget, the method creates the user s carbon composite table to be used later in creating the carbon composite layer. GetBarChartData() Executed from the Bar Chart widget, the method returns carbon values for a single or set of pre-identified data sets for a given county, state, or region selected by the user in the map. Results are displayed in the Bar Chart Tool popup window. GetCarbonVisualizationSetting() Executed from the User Preferences widget, the method returns the user s preferences for charting, carbon identify, and generating the carbon composite layer. GetCounties() Executed from the Find widget, the method returns a unique list of counties within a specified state. GetDataScale() Executed from the Model Selection widget, the method returns a unique list of model data scale types. GetDataSetInfo() Executed from the More Info widget, the method returns information about a specific data set. GetDistinctScenarioDates() Executed during the mapping application load, the method returns a list of distinct scenario dates for each carbon data set. GetFedLands() Executed from the Find widget, the method returns a unique list of federal lands (e.g., national parks). GetIdentifyData() Executed from the Carbon Identify widget, the method returns carbon values for a single or set of pre-identified data sets for a given county, state, or region selected by the user in the map. Results are displayed in the Carbon Identify Tool popup window. GetLandCover() Executed from the Model Selection widget, the method returns a unique list of model land cover types. GetMapLayers() Executed during the mapping application load, the method returns a list of the map layers available to the user. GetModelScenario() Executed from the Model Selection widget, the method returns a list of model scenarios available to the user. GetModelWithAttributes() Executed from the Model Selection widget, the method returns a list of model scenarios with attributes (i.e., land cover, time step, and data scale). Page 19 of 64

20 GetModelByAttributes() Executed from the Model Selection widget, the method returns a list of model scenarios with the selected attributes (i.e., land cover, time step, and data scale). GetModelInfo() Executed from the More Info widget, the method returns information about a specific carbon model. GetPieChartTable() Executed from the Pie Chart widget, the method returns carbon values for a single or set of pre-identified data sets for a given county, state, or region selected by the user in the map. Results are displayed in the Pie Chart Tool popup window. GetScenarioInfo() Executed from the More Info widget, the method returns information about a specific carbon model scenario. GetStates() Executed from the Find widget, the method returns a unique list of states. GetTimeSeries() Executed from the Time Series widget, the method retrieves a list of time series map layers and attributes available to the user. Get TimeSeriesInfo() Executed from the More Info widget, the method returns time-specific information (e.g., start and end dates) about a specific time-aware data set. GetTimeStep() Executed from the Model Selection widget, the method returns a unique list of model time step types. GetUserCarbonComposite() Executed from the User Preferences widget, the method retrieves the carbon values saved in the user s carbon composite table to create the carbon composite layer. GetUserMapLayers Executed from the main application (.NET), the method retrieves the map layers and associated information available to the suer. GetUserUnits() Executed from the User Preferences widget, the method retrieves the user s preference for display unit. The default setting is g/m2. GetUSFSRegions() Executed from the Find widget, the method returns a unique list of U.S. Forest Service regions. SetDefaultCarbonVisualizationSetting() Executed from the User Preferences widget, the method reverts (i.e., saves) the user s preferences for charting, carbon identify, and generating the carbon composite layer to the default settings. SetUserUnits() Executed from the User Preferences widget, the method sets (i.e., saves) the user s preference for display unit. UpdateCarbonVisualizationSetting() Executed from the User Preferences widget, the method updates (i.e., saves) the user s preferences for charting, carbon identify, and generating the carbon composite layer Private Functions (Called from within.net web service) GenerateUserCarbonCompositeTable_Internal() Called from GenerateUserCarbonCompositeTable. The function creates the user s carbon composite table to be used later in creating the carbon composite layer. GetChartFieldsTable() Called from GetBarChartData() and GetPieChartTable(). The function returns the carbon fields for a given default or user selected carbon data set identified in the User Preferences window for charting (i.e., return results option). Page 20 of 64

21 GetChartFieldsDataTable() Called from GetBarChartData() and GetPieChartTable(). The function returns the carbon data for a given default or user selected carbon data set identified in the User Preferences window for charting (i.e., return results option). GetChartColorTable() Called from GetBarChartData() and GetPieChartTable(). The function returns the pre-defined chart color for the carbon field. Evaluate() Called from GetIdentifyData(), GetBarChartData(), and GetPieChartTable(). The function converts a string expression of a number to its double equivalent and returns the double. 2.2 Sql Server Databases Two enterprise-level Microsoft SQL Server databases are being used to maintain the data for the CDVT- P: 1) non-spatial (sql_usda) and 2) spatial (sde_usda). Additionallly, Microsoft SQL Server Express is being used to manage the security for the map services being published by ArcGIS Server. The following sections describe each database in detail Spatial Database When storing and serving up geospatial data to ESRI s ArcGIS Server, data can be stored in various formats ranging from shapefiles to single user databases (file and personal) to RDBMS (relational database management system) focused multi-user databases. Storing data in an RDBMS database structure offers the end user the ability to: Store a rich collection of spatial data in a centralized location Apply sophisticated rules and relationships to the data Define advanced geospatial relational models (e.g., topologies, networks). Maintain integrity of spatial data with a consistent, accurate database. Work within a multiuser access and editing environment. Integrate spatial data with other IT databases. Easily scale your storage solution. Support custom features and behavior. Leverage your spatial data to its full potential. By utilizing ESRI s ArcSDE technology, it allows for management of spatial data in an RDBMS and enables it to be accessed by ArcGIS clients (including ArcGIS for Server). This technology also provides the framework to support long transactions, which facilitates the versioned editing environment in multiuser databases Model Scenario Data Sets Application-specific Fields The USDA Carbon Model requires some specific fields throughout the data to utilize some of the built in tools within the Flex viewer: COUNTYFP (Polygon) For data at the county level, the COUNTYFP field must be filled out with the correct data. This allows for tools like the pie and bar chart to correctly display the related data for the county. Page 21 of 64

22 Time-Aware STATEFP (Polygon) For data at the state level, the STATEFP field must be filled out with the correct data. This allows for tools like the pie and bar chart to correctly display the related data for the state. USFSRegion (Polygon) For data at the US Forest Service region, the USFSRegion field must be filled out with the correct data. This allows for tools like the pie and bar chart to correctly display the related data for the region. GM2 Carbon Fields (Polygon) The GM2 Carbon fields for each data layer must be specified within the user preferences data tables within the business table s database. Period The User Preferences pane and Time Series tool within the Carbon Model application uses this field for filtering the data by date. For any data layers to be utilized within the time series tool, the period (or period_ts) field is utilized. The _TS on Period stands for time series if the time data needs to be at a refined level compared to period Carbon Field Mapping The carbon composite (graphics) layer is generated from: 1. A primary feature class (CarbonComposite) containing all the U.S. county boundaries (polygons). 2. A set of pre-defined, default carbon fields as identified by the DefaulAddend field in the modelcarbonpoolfieldmapping table in the non-spatial database OR A user defined business table containing county FIPS codes and the carbon fields and associated carbon values from dictated by user selections made in the User Preferences window. The naming convention for the user defined business tables is CarbonComposite followed by an underscore and then the user s unique identifier for the system (e.g., CarbonComposite_3c56ef9ad90a495f85c b44e). User defined business tables are created in the spatial database when the user overrides the default carbon composite fields Carbon-Related Data Sets The following layers are considered carbon-related data sets. This list is generated from the business tables by querying ContainsCarbon > 0 from the vmaplayers SQL view. The column is housed in the dbo.dataset table within the business table database. Data Set Name ESRI ArcSDE Feature Name Page 22 of 64

23 Carbon Calculation Tool - EPA Submission CCT_STATE_ALL_STOCK_PROJECT Forest Inventory and Analysis Forest Carbon Budget Model 4241 Forest Carbon Budget Model 4242 Forest Carbon Budget Model 4251 Forest Carbon Budget Model 5101 Forest Carbon Budget Model 5102 Century Model - Lexington CT0N Century - Lexington NT8N CQESTR Lexington CT0N CQESTR Lexington CT336N SSURGO Derived Soil Organic Carbon 0-30cm SSURGO (USDA R&D gridded version) 0-100cm Carbon Calculation Tool (County) Carbon Composite Forest Above Ground Carbon Forest Below Ground Carbon Forest Floor Carbon Forest Soil Organic Carbon Forest Standing Dead Carbon Forest Total Carbon Forest Understory Above Ground Carbon FIA_WMSPHERE_CONVERT FORCARB2_4142_Rollup_GM2 FORCARB2_4242_Rollup_GM2 FORCARB2_4251_Rollup_GM2 FORCARB2_5101_Rollup_GM2 FORCARB2_5102_Rollup_GM2 Century_Lexington_CT0N Century_Lexington_NT8N CQESTR_Lexington_CT0N_V2 CQESTR_Lexington_CT336N_V2 SSURGO_County_Rollup_Final US_SSURGO_ALBERS_JOINED2 CCT_Counties Carbon_Composite CARBON_ABOVEGROUND CARBON_BELOWGROUND CARBON_FOREST_FLOOR CARBON_SOIL_ORGANIC CARBON_STANDING_DEAD CARBON_TOTAL CARBON_UNDERSTORY_ABOVEGROUND Page 23 of 64

24 Forest Understory Below Ground Carbon Forest Down Dead Carbon CARBON_UNDERSTORY_BELOWGROUND CARBON_DOWN_DEAD Application-specific Fields Figure 2.17: Carbon-related Data Sets The USDA Carbon-Related Data requires some specific fields throughout the data to utilize some of the built in tools within the Flex viewer: COUNTYFP (Polygon) For data at the county level, the COUNTYFP field must be filled out with the correct data. This allows for tools like the pie and bar chart to correctly display the related data for the county. STATEFP (Polygon) For data at the state level, the STATEFP field must be filled out with the correct data. This allows for tools like the pie and bar chart to correctly display the related data for the state. USFSRegion (Polygon) For data at the US Forest Service region, the USFSRegion field must be filled out with the correct data. This allows for tools like the pie and bar chart to correctly display the related data for the region. GM2 Carbon Fields (Polygon) The GM2 Carbon fields for each data layer must be specified within the user preferences data tables within the business table s database. Period The User Preferences pane and Time Series tool within the Carbon Model application uses this field for filtering the data by date Reference Data Sets The application allows for reference datasets to be used to help define and divide the data into various footprints and regions. These layers consist of: States Footprints for the continental U.S. (Has been simplified to remove erroneous vertices from the polygon and to speed up performance over slower connections). USFS Regions Regions previously defined by the US Forest Service Application Dependent Data Sets The application depends on a few services from ESRI ArcGIS Server to properly function. These layers provide the required query attributes needed to perform spatial and data queries against the business tables and other related services. These layers consist of: Counties This layer is the initial layer shown when launching the Carbon Model application. It is also the base layer you will click on with the Bar chart and Pie chart tool to select county level information. (Has been simplified to remove erroneous vertices from the polygon and to speed up performance over slower connections). Data Availability This service consists of 1 feature class called Availability and is housed on the ESRI ArcSDE database. It contains a one-to-many relationship of polygon footprints that related to the services for spatial selection. The DataSetID is used in the business tables and can be used by many layers. Page 24 of 64

25 2.2.2 Non-Spatial Database The business tables contained within the non-spatial SQL Server database are organized for the most part by functional group. For instance, the data sets (carbon model scenario, carbon-related, and reference map layers) available in the CDVT-P are stored in the DataSet table. Related domain values, such as the data set category or map service type, are stored in domain tables prefixed with dataset. Additionally, security-related tables are prefixed with the default aspnet when using ASP.NET Membership as well as user and role respectively Business Tables Provided below is the high-level database structure for the non-spatial tables used for the main (ASP.NET) and mapping applications. Tools Figure 2.18: Mapping Application High-Level Database Structure Based on a user s application role, CDVT-P tools may be enabled (visible) or disabled (hidden) upon load of the application. The Tool table contains the unique identifier, name, and description for all CDVT-P tools (both administrative and mapping). Note: In order to manage CDVT-P mapping tools by application role, the mapping application s configuration file (config-amazon.xml) contains a subset of the tools specific to the mapping application. Data Sets The DataSet table contains all the data sets (model scenario, carbon-related, and reference map layers) available in the CDVT-P. More specifically, the DataSet table contains information about each data set including but not limited to the label displayed in the application, the map service, the underlying feature class, various metadata attributes (e.g., source, abstract), the related data set availability footprint, and its participation in the various mapping tools (i.e., is the data set used in charting, identify, carbon composite, Page 25 of 64

26 etc.). All data set related domain values are contained in domain tables prefixed with dataset. Data set related domain tables are: datasetaggregatelevel - Contains the unique values for the data set aggregate level or the geographic unit used to represent the data set (e.g., USFS region, state, county, or stand/point). datasetappusecategory - Contains the unique values for the data set application use category or how to categorize the data set (e.g., model scenario, carbon-related data, or reference data). datasetdatatype - Contains the unique values for the data set data type (e.g., vector, raster, or tabular). dataseticon - Contains the unique values for icons used in CDVT-P (e.g., the map layer list) to represent the application use of the data set (e.g., model scenario, carbon-related data, or reference data). datasetmapservicetype - Contains the unique values for ArcGIS Server map service types used in CDVT-P (e.g., dynamic or tiled). The TimeSeries table contains all the time-aware data sets available in the CDVT-P. More specifically, the TimeSeries table contains information about each time-aware data set including but not limited to the corresponding data set (id), map service, from date, to date, and time interval. The FromDate, ToDate, TimeInterval and TimeIntervalUnits fields are used in the execution of the time series. Models, Scenarios, and Carbon Pools The Model table contains all the carbon estimating models available in the CDVT-P. More specifically, the Model table contains information about each carbon estimating model including but not limited to the label displayed in the application for the carbon estimating model, description of the carbon estimating model, and technical specifications of the carbon estimating model. The modelmodelagency table contains the agencies (id) for the carbon estimating models. A unique list of agencies can be found in the Agency table. The modeldocumentation table contains a description and the file path or website url for documentation about the carbon estimating models. The modelmodellandcover table contains the land cover types (id) for the carbon estimating models (e.g., agriculture, forest, or rangeland). A unique list of land cover types can be found in the modellandcover table. The modelinputsample table contains samples of input to the carbon estimating models. The modeloutputsample table contains samples of output from the carbon estimating models. The modelinputtimestep table contains the time steps used as input to the carbon estimating models (e.g., single point in time or multiple points in time historical). Page 26 of 64

27 The modeloutputtimestep table contains the time steps outputted from the carbon estimating models (e.g., single point in time or multiple points in time future). All carbon estimating model related domain values are contained in domain tables prefixed with model. Carbon estimating model related domain tables are: modeldataformat - Contains the unique values for the carbon estimating model input and output data formats (e.g., ASCII or raster). modeldatascale - Contains the unique values for carbon estimating model scales or geographic units (e.g., USFS region, state, county, or stand). modellandcover - Contains the unique values for carbon estimating model land cover types (e.g., agriculture, forest, or rangeland). modeltimestep - Contains the unique values for carbon estimating model time steps (e.g., single point in time or multiple points in time - historical). modelunittype Contains the unique values for carbon estimateing model unit types (e.g., g/m2). The Scenario table contains all carbon model scenario results available in the CDVT-P. More specifically, the Scenario table contains the carbon estimating model (id) and data set (id) associated with the carbon estimating model scenario result. Model and data set information are stored in the Model and DataSet tables respectively. The modelcarbonpool table contains a unique list of carbon pools in CDVT-P. The CarbonPoolDetail field provides the application display name for the carbon pool. The ChartColor field provides the display color (hex code) for the carbon pool to be used in bar or pie charting. The modelcarbonpoolfieldmapping table contains all the carbon pool field mappings for carbon data sets in CDVT-P (e.g., FORCARB2 or SSURGO). More specifically, the modelcarbonpoolfieldmapping table contains information the data set (id), the carbon pool field (id), and the feature class attribute field name. Data set and carbon pool field information are stored in the DataSet and modelcarbonpool tables respectively. In addition, the modelcarbonpoolfieldmapping table contains default/public user settings for charting and the carbon composite layer (i.e., is the carbon pool field by default selected for charting or included in the carbon composite layer?). The modelcarbonpoolcategory domain table contains unique values for carbon pool types (e.g., above ground, below ground, or total). The modelcarbonvisualbasicsetting table contains the data set level (i.e., the parent node level) userdefined/saved selections for carbon visualization in the User Preferences window. More specifically, the modelcarbonvisualbasicsetting table contains the data set (id), preferred scenario date, and whether or not to return the data set results in charting and identify for the user (id). Data set information is stored in the DataSet table. The modelcarbonvisualdetailsetting table contains the carbon pool field level (i.e., the child node level) user-defined/saved selections for carbon visualization in the User Preferences window. More specifically, Page 27 of 64

28 the modelcarbonvisualdetailsetting table contains the data set (id), the carbon pool field (id), whether or not to return the carbon pool field in charting and identify, or whether or not to include the carbon pool field in the carbon composite layer for the user (id). Data set and carbon pool field information are stored in the DataSet and modelcarbonpoolfieldmapping tables respectively. Miscellaneous User Preferences The userlayerprefs table contains the map layer user preferences for the layer list. More specifically, the UserLayerPrefs table contains the user preferences for layer visibility, layer opacity, and load order for each data set (id). Data set information is stored in the DataSet table. Presently, the user preferences for map layers can only be set manually in the database table. The userunitpref table contains the display unit preference for each user. The default setting is metric. The userunittype domain table contains a list of unique unit types that may be used in CDVT-P. Content Management Some of the CDVT-P content can be edited by privileged users. This content is stored and managed in the following tables: The Message table contains the welcome message for non-authenticated users, the welcome message for authenticated users, and contact information that appear in the Login and Welcome pages. The messagetype table Contains the unique values for the message type (e.g., Contacts, Welcome Message for Authenticated Users, Welcome Message for Non-Authenticated Users ). The Link table contains the links that appear at the bottom of the Login and Welcome pages. linktype contains the unique values for the link type (e.g., Carbon Model Information, Forest Data, Geospatial Data ). The LookupTable table contains metadata about the lookup tables that can be edited by privileged users. The lookup table content is stored in the table specified in the LookupTable metadata (e.g., linktype, modelunittype) Security Tables The security tables contained within the non-spatial SQL Server database consist of some ASP.NET Membership t Provider tables and some custom tables. Standard user information is stored in the ASP.NET Membership Provider tables, and custom user attributes and role-based security settings are stored in custom tables. The following illustration presents the database structure for the security tables. Page 28 of 64

29 Figure 2.19: Security High-Level Database Structure Users All of the ASP.NET Membership Provider table names begin with aspnet. The aspnet_users table contains the unique User ID and User Name for all CDVT-P users. Password, Address, and Last Login Date are amongst the standard user attributes that are stored in the aspnet_membership table. User attributes that are custom to the CDVT-P application are stored in the userdetails table such as First Name, Last Name, Organization, Title, and Role. aspnet_applications contains just one record for the CDVT-P application. aspnet_schemaversions contains metadata that determine the structure of the ASP.NET Membership Provider schema. Roles The userrole table contains all of the CDVT-P user roles recognized by the application. Based on a user s application role, CDVT-P tools and datasets may be enabled (visible) or disabled (hidden) upon load of the application. The roletools and roledatasets tables store these settings. The Tool table contains all CDVT-P tools, and the DataSet table contains all CDVT-P datasets SQL Express It was determined that some map services required security in order prevent other organizations from consuming these resources which was accomplished by enabling security on ArcGIS Server. The options for SQL Server express was used to achieve this level of security since it was necessary to allow public as well secured map services. This solution is also maintained through ArcGIS Server Manager which is a standard tool to manage map services. The SQL Express database is controlled through the ArcGIS Server Manager application which can create new users and apply permissions to map service. The SQL Express database is a membership database. Example of applying permission: 1. Log into ArcGIS Server Manager (typically Page 29 of 64

30 2. Go to Services tab click on Manage Services Figure 2.20: Manage Services Tab 3. Choose folder or map service in root and click on Permissions. Figure 2.21: Manage Folder Options 4. Add the desired roles to map service or folder. Figure 2.22: Permissions Mapping Reference: ent_security_in_arcgis_server/ p / 2.3 Map Services ESRI ArcGIS Server allows for dynamic and cached map services, among many other features. The USDA Carbon Model application utilizes these services to provide the data in a graphical representation displayed to the end user. Page 30 of 64

31 Note: ESRI ArcGIS Server has a limited number of service and ArcSDE instances supported on Windows Servers. If experiencing issues, please refer to and Directory Structure & Security To utilize security at the user login level, ESRI ArcGIS Server allows for services to belong to security groups that are managed at a virtual folder level. For ease of management within ESRI ArcGIS Server, the services are split into two folders: Common Any services that should always be available to any end-user and considered public information (EG: county borders, water bodies, cities, roads, etc.) Secure Services that contain data that is considered sensitive or restricted (EG: CDL change detection, federal lands, carbon models, etc.). Please see Section 4.2 for ArcGIS Server configuration. Example of the folder structure when viewed via the web-based interface for ESRI ArcGIS Server. Figure 2.22: CDVT-P Map Service Folder Structure in ArcGIS Server Manager Example of the folder structure when viewed via ESRI ArcCatalog Dynamic Map Services Basic Map Services Figure 2.23: CDVT-P Map Service Folder Structure in ArcCatalog Basic layers are map services that do not include time-enabled layers or rasters within the MXD document. Time-enabled layers should be grouped together in separate MXDs when there are multiple layers and rasters should be saved separately in MXDs with no other data for caching purposes. By grouping vector datasets that do not require caching into one service, the requirement for active connections and services are greatly reduced (see note under 2.3.1). By grouping like-data into one service, data management becomes easier as you can isolate data-locks. Page 31 of 64

32 An example of vector data that can be grouped into one service: Counties Data Availability States Major Roads Cities Bodies of Water Watersheds Time Series Map Services Time Series layers are map services that are time-enabled layers within an MXD document. When an MXD is published to ESRI ArcGIS Server, all layers contained within the document are processed and any time layers are automatically flagged. There are two ways to verify if a service is time enabled: Within the REST endpoint: 1. Using a web browser, navigate to the REST services (e.g.: name]/arcgis/rest) and login using the username/password that has been previously specified within ESRI ArcGIS Server (this is not the same as a login into the Carbon Model application). 2. Drill down into the secure services and select a service. For ease of use by the developers, all services within the application have _TimeSeries appended to the end of the service name. 3. Within the service, scroll towards the bottom and you should see the following: Time Info: Time Extent: [1990/01/01 00:00:00 UTC, 2012/01/01 00:00:00 UTC] Time Reference: N/A This information is read in Year/Month/Day Time. Within ArcGIS Desktop: 1. Open the MXD. 2. Double click on a layer or right click and select properties. 3. Navigate to the Time tab. If the checkbox is checked, the layer is time enabled Carbon Composite Map Service The carbon composite is a dynamic service that returns 3,500 records when queried. ESRI ArcGIS Server has a default value of 1,000 records and requires the configuration file to be modified every time the Carbon Composite layer has been republished to the server. Starting and stopping the service does not require this step; only when the configuration file has been overwritten. To change the default value: 1. Open C:\Program Files (x86)\arcgis\server10.0\server\user\cfg\common\carbon_composite.mapserver.cfg using Notepad as an Administrator. 2. Search for <MaxRecordCount>1000</MaxRecordCount> (Usually around Line 10) 3. Change the value of 1000 to Page 32 of 64

33 4. Save and close. 5. Restart IIS Cached Map Services Cached map services are designed to minimize the latency between the request for a spatial extent and the data contained within it. Map services are able to provide minimal latency by creating a stockpile (or cache) of tiles that can be downloaded on demand instead of dynamically. By having tiles cached, the server can request surrounding tiles in the background increasing the speed at which the end user can pan and zoom the map. Creating the caches requires the server to go tile by tile writing the data to JPGs for future use. Please see Section on how to create a cache. Benefits: Better performance - almost instantaneous serving of large datasets once initial cache has been created. Can isolate the extent that need to be recreated if missing or recreating tiles due to symbology changes. During downtime afterhours and weekends, more server resources can be utilized to minimize cache creation. If zooming to an extent that hasn t yet had cache s for, caches will be generated dynamically (though with a performance hit to the server). Disadvantages: Requires a file type of PNG32 if transparency is needed. Initial preparation time is required for generating large caches. Requires large amount of disk storage for caches. If zooming to an extent that hasn t yet had caches for, tile creation can put a strain on the server for everyone else Third Party Map Services The Carbon Model application can take advantage of third party map services to be referenced and consumed. These services can be aerial backgrounds, vector and raster data. None of the native tools to the Carbon Model application can return data from these unless the services are setup within the business tables and follow this guide. An example of third party map services currently in use for this application are the aerial background provided by ESRI for public consumption. Page 33 of 64

34 Section 3. Solution Administration 3.1 Software Application Management Managing Model Information 1. Log in as an administrator. 2. Select the Model Information link on the site navigation bar. Figure 3.1: Model Information Link The Model Information page displays the models used within the Carbon Data Visualization Mapping Module. The models are aligned in a list and a tabular format. Figure 3.2: Model Information Summary Page 3. Select either the model name from the left Model list or from the center table to view the model information specific to the selected model. Page 34 of 64

35 Edit links will be displayed next to model information data elements to allow users with Administration rights to modify the data. Figure 3.3: Model Information Page Update Model Label, Name, and Description 1. Click the Edit link next to Description. The Update Model Information page is displayed. Figure 3.4: Update Model Information Page 2. Make appropriate changes and click the Update Model link to save changes. Page 35 of 64

36 Update Model Documentation 1. Click the Edit link next to Documentaion. The model document/link title summary displays. Figure 3.5: Edit Model Documentation 2. Click the Edit link next to the existing document/link title. The model document/link title page displays. Figure 3.6: Update Model Documentation Page 3. Update the document/link title and click Save Add Model Documentation 1. Click the Edit link next to Documentaion. The model document/link title summary displays. 2. Click Add New Document/Link. Figure 3.7: Add Model Documentation The Model Documetation page displays. Page 36 of 64

37 Figure 3.7: Add Model Documentation Page 1. Select the documentation type. 2. Enter the document title. 3. Browse to/enter the document file or url path. 4. Click Save Delete Model Documentation 1. Click the Edit link next to Documentaion. 2. The model document/link title summary displays. Figure 3.8: Delete Model Documentation 3. Click the Delete link next to the existing document/link title Update Model Scenarios 1. Click the Edit link next to Scenarios. 2. The model scenario summary displays Add Model Scenarios 1. Click the Edit link next to Scenarios. 2. The model scenario summary displays Delete Model Scenarios 1. Click the Edit link next to Scenarios. 2. The model scenario summary displays. Page 37 of 64

38 Update Model General Information 1. Click the Edit link next to General Information. 2. The Update Model Information page is displayed Update Model Input 1. Click the Edit link next to Input. 2. The Update Model Information page is displayed Update Model Output 1. Click the Edit link next to Output. 2. The Update Model Information page is displayed. Page 38 of 64

39 3.1.2 Managing Map Layer Information 1. Select the Map Layer Information link on the site navigation bar. 2. Select category to choose layer for update. 3. Click Edit button to activate editing session. 4. An edit form will appear that will allow the user to change values. a. To replace Legend click checkbox Replace Legend Image b. A file locator will be presented to user 5. To save edits, click Update map Layer button. Page 39 of 64

40 3.1.3 Managing Users and Roles CDVT-P users and roles are managed by privileged users through the CDVT-P user interface. For detailed instructions on how to manage users and roles, please refer to Section 8.1 (Managing Users) and Section 8.2 (Managing Roles) of the USDA Carbon Data Visualization Tool Software Users Manual. Also, much of the CDVT-P content can be edited by privileged users through the CDVT-P user interface. This content includes model information and supporting documentation, map layer information, welcome page content, links, and lookup tables. For detailed instructions on how to manage CDVT-P content, please refer to Section 5 (Model Information Module), Section 6 (Map Layer Information Module), Section 8.3 (Managing Welcome Page Content), Section 8.4 (Managing Links), and Section 8.5 (Managing Lookup Tables) of the USDA Carbon Data Visualization Tool Software Users Manual. 3.2 Database Management The CDVT-P application data and settings have been designed to be housed in an relational database management system model (RDBMS) served up by a enterprise-level database server. Currently, this is housed on Microsoft SQL Server 2008 R2 and this user guide is only written with this in mind. The spatial data has been separated from the business tables and housed in two separate databases for ease of management. The spatial data should be managed using Esri s ArcGIS tools (mainly ArcMap and ArcCatalog) while the business tables should be managed using Microsoft SQL Server Management Studio Spatial Data Management By utilizing an RDBMS model, the CDVT-P application utilizes spatial data housed in Esri s ArcSDE with a Microsoft SQL Server backend. ArcSDE allows for enterprise level backups, performance, user isolation and many other features. To manage the spatial data, the end user needs to interface with the data using Esri s ArcGIS Desktop software, specifically ArcCatalog. Esri s ArcGIS Desktop is required for managing CDVT-P spatial data. The following applications and tools are needed: ArcGIS Catalog ArcToolbox Server Tools\Caching Create Map Service Cache Manage Map Server Cache Tiles New Map Layers Page 40 of 64

41 The following sections will walk an system administrator through the necessary steps to add spatial data to ESRI ArcGIS Server as new services to be consumed by ESRI ArcGIS Desktop and the CDVT-P. It will cover data preparation, service creation and cache creation. When new data is ready to be brought into the application, it must first go through some data preparation to include the appropriate fields within the dataset. Once loaded into SDE using ESRI ArcCatalog, make sure to do the standard data administration tasks to the data. The standard data administration tasks include: Registering data as versioned (Polygon only) - if data will not be used in a working environment and only to be referenced within the Carbon Model application, check the Register the selected objects with the option to move edits to base. to allow the database to see data without the database being compressed to a state of zero. Analyze - check all available options. Build Pyramids (raster only). Calculate Statistics (raster only) Default ESRI ArcGIS options are fine. For required application specific fields, please see Section ArcGIS Server Map Service Creation 1. Create map content. Using ArcMap, load all data needed for service into MXD. Rename the layers accordingly in the TOC (remove any SDE naming schema; IE: remove sde_usda.dbo.) and symbolize as needed. Set the MXD coordinate system to WGS 1984 Web Mercator (Auxiliary Sphere) even if your data is in a different projection. Note: The file location of the CDVT-P MXDs/MSDs on AmazonAGS is D:\arcgisserver\arcgismxds. 2. Set map scales. Change the scale settings to match that of the aerial used for cache creation. Once the scales have been set to the ArcGIS Online services, set the default scale to be 1:36,978,595. Make sure that your data is panned to show the continental United States. Page 41 of 64

42 Figure 3.1: Map Scale Settings 3. Publish map to ArcGIS Server. Save your MXD to the appropriate folder depending on your server as the name of the service to be used in Flex. Using the Map Service Publishing toolbar, click the 4th icon from the left that says Publish to ArcGIS Server. Figure 3.2: Publish to ArcGIS Server Select the existing folder that the related service should reside in. Press next. If republishing over an existing service, select yes to replace it. Uncheck KML from the list and select finish. Note: The purpose of using an existing folder is for security purposes within the Flex app. The Flex app utilizes.net authentication on the IIS backend and will isolate data services for authorized users. Figure 3.3: Publish Map Service Dialog Page 42 of 64

43 4. Clear ArcGIS Server cache. After all services have been created, sometimes the cache needs to be cleared before they will show up in the REST interface. Log into name]/arcgis/rest/admin using your credentials and select Cache. Select Clear Cache Now to reset the cache. This does not clear out any map cache s residing on the server, just anything that is residing in the memory ArcGIS Server Cache Creation Numerous CDVT-P map services are cached to improve system performance. Relying on the base map, the source for the applied tiling scheme level of detail (LOD) is (ESRI World Imagery). Create Map Service Cache Tool Set the tool settings to match the below screenshots. Hit Ok to start the geoprocessing service. The process will create a local holding location for any caches created and register the cache with the ArcGIS Server. Page 43 of 64

44 Figure 3.4: Create Map Service Cache Manage Map Server Cache Tiles Set the tool settings to match the following screenshots. Set the cache levels to match the following levels of detail. Expected time for the process to run will be roughly 5 to 6 hours. Any caches created can be cross-transferred between local development servers and Amazon via FTP (though the Create Map Service Cache tool needs to be ran on each server before the data is transferred) Source of Tiling Scheme LOD: (ESRI World Imagery) Zoom Level ArcGIS Online Scale Scale (1" = Ft) Processing Time (minutes) Processing Time (hours) Estimated Disk Size 0 591,657, ,304, Kb 1 295,828, ,652, KB 2 147,914, ,326, Kb 3 73,957, ,163, Kb 4 36,978, ,081, Kb 5 18,489, ,540, Kb 6 9,244, , Mb 7 4,622, , Mb Mb Page 44 of 64

45 2,311, , ,155, , Mb , , Mb , , Mb , , Gb Figure 3.5: Initial Levels of Detail To Create Figure 3.6: Manage Map Server Cache Tiles Note: The file location of the CDVT-P tiling scheme template (and cache folders) on Amazon is D:\arcgisserver\arcgiscache\WGS84_WebMercator_AuxSphere_TilingScheme.xml. Page 45 of 64

46 Updated Map Layer When updating map layers, always work with within the MXD that was originally created for the service. This will minimize issues regarding Time Series and other settings that may have been predefined when previously setting up the service. If swapping out data layers within a service with fresh ones, please make sure the same fields are in the new raster/feature classes and that they match. Any changes to the carbon related services need to be reflected in the business tables before publishing to the server for the tools to work correctly. Updating map layers is similar to creating new dynamic map layers. Map layers that have been cached will require a full recreation of the tiles if the symbology has changed for raster and vector data. Attribute changes do not affect cached map layers as long as the symbology is not based on the attribute field in question. If a change has been applied to the service and republished, sometimes the REST API cache and your local browser cache needs to be cleared before the settings are displayed. To clear the REST API cache, see step 5 under Section To create a new field and populate with data: 1. Prior to making any field changes, stop ArcGIS Server (SOM and SOC) on web server. Note: If a non-versioned database, only the map service needs to be stopped. Page 46 of 64

47 2. Launch ArcCatalog and browse to data location in SDE (requires a spatial database connection). 3. Right click on feature class and select Properties. 4. Select Fields tab. Page 47 of 64

48 5. Create field. 6. Click Apply and OK. 7. Launch ArcMap and load feature class. 8. Start editing (may require the feature class to be registered as versioned). 9. Populate data for new field. 10. Save edits and stop editing. 11. Close ArcMap. 12. If a versioned database, compress SDE database. See Start ArcGIS Server (SOM and SOC) on web server. Note: If a non-versioned database, restart map service. 14. Verify field and data update through Rest API or application (identify tool). Page 48 of 64

49 To populate or update data in an existing field: 1. Prior to making any field changes, stop ArcGIS Server (SOM and SOC) on web server. Note: If a non-versioned database, only the map service needs to be stopped. 2. Launch ArcMap and load feature class. 3. Start editing (may require the feature class to be registered as versioned). 4. Populate / update data. 5. Save edits and stop editing. 6. Close ArcMap. 7. If a versioned database, compress SDE database. See 8. Start ArcGIS Server (SOM and SOC) on web server. Note: If a non-versioned database, restart map service. 9. Verify field and data update through Rest API or application (identify tool) Availability Feature Class The Availability feature class contains polygons that represent the area of coverage for a carbon model which is used in the Data Availability / Selection tool. The DataSet Table field AvailabilityDataSetId corresponds to Availability feature class field DataSetId in which many carbon models can be related to one Availability polygon. Page 49 of 64

50 Steps in adding a new polygon carbon model footprint: 1. Create new polygon and add to Availability feature class, filing in the next value in DataSetId attribute field. 2. Open DataSet table in SQL Server manager, edit the corresponding models AvailabilityDataSetId to match the value from step Time Series Map Layers Esri ArcGIS allows for temporal data to be represented visually both within ArcMap and ArcGIS Server. By loading temporal data into ArcMap, the end user can setup a timeline for the data to be shown and publish to ArcGIS server. Steps in time-enabling a layer: 1. Load a map layer into ArcMap that has temporal data in it (for CDVT-P, all dates are stored in the PERIOD text field). 2. Open the properties of the layer and select the Time tab. Check the Enable time on this layer to identify the layer as temporal. 3. Set the fields in this window appropriately. For CDVT-P, each record in the data was represented with a single date (versus a date range). Dates are stored in a YYYYMMDD format (year month day). Page 50 of 64

51 Figure 3.7: Map Layer Time Properties 4. On the Tools toolbar, click the icon that looks like it has a little clock on it to bring up the Time Slider. By playing with the time slider properties, you can adjust what data is represented by the current time/date and display settings. Figure 3.8: Time Tools and Slider 5. Once the temporal data is operating as expected, you will publish the services using the same method in Section Non-spatial Data Management To manage the non-spatial data (also known as the business tables), you must connect to the SQL Server instance using Microsoft SQL Server Management Studio (also known as SSMS). All business tables are stored in a database titled sql_usda. SSMS is also what you will use to create any backups or Page 51 of 64