EPIRB Web GIS Portal Training Minsk, Republic of Belarus 25-26 July 2016 Lyubomir Filipov.
Course introduction 1. Project activities introduction from Deputy Team Leader of EPIRB Project, Mr. Zurab Jincharadze; 2. Instructor introduction: educational background and professional experience. 3. Participants introduction: - Name; - Organization; - Role in organization; - GIS experience; - Goals and expectations for the training.
Course timeline 1. Day one: 09:00-17:00 Course introduction; Exploring GIS concepts; GIS on the Web; EPIRB system architecture; 2. Day two: 09:00-17:00 Editing data; Presenting data; Exersices, Q&A.
Q&A visit: https://www.sli.do/ event code: 6922
Exploring GIS concepts
Exploring GIS concepts: Definitions of GIS Aronoff (1989): any manual or computer based set of procedures used to store and manipulate geographically referenced data. Carter (1989): an institutional entity, reflecting an organizational structure that integrates technology with a database, expertise and continuing financial support over time.
Exploring GIS concepts: Definitions of GIS Cowen (1988): a decision support system involving the integration of spatially referenced data in a problem-solving environment. ESRI: A geographic information system (GIS) integrates hardware, software, and data for capturing, managing, analyzing, and displaying all forms of geographically referenced information.
Exploring GIS concepts: History of GIS Pioneer stage: end of 50 s and beginning of 60 s; Creation of first commercial GIS software 70 s; Development of separate market segment mid 90 s; Fast commercialization 2005; Web, mobile, social - now.
Exploring GIS concepts: Need of GIS Easy exchange of information between different users; Decision support system; Different application of the end product; Quick data capturing, manipulation and analysis.
Exploring GIS concepts: GIS functions Capture Store Query Analyze Display Output
Exploring GIS concepts \ GIS functions: Capturing data
GIS functions: Storing data Vector formats: Discrete representations of reality (X4,Y4) (X3,Y3) (X,Y) Raster formats: Use square cells to model reality (X5,Y5)
Exploring GIS concepts \ GIS functions: Queries Identifying specific features Identifying features, based on conditions
Exploring GIS concepts \ GIS functions: Analysis Proximity Overlay Network
Exploring GIS concepts \ GIS functions: Modeling
Exploring GIS concepts \ GIS functions: Display maps, graphs, reports
Exploring GIS concepts \ GIS functions: Output Internet Internet Paper Paper map map GIS Data Document Document Image Image Admin.qgs UAPrut_HMWB_A WB.jpg
Exploring GIS concepts: Components of GIS Experts Data Hardware Software GIS Procedures UAPrut_HMWB_A WB.jpg
Exploring GIS concepts: Other topics Coordinate systems and map scale: geographic coordinate systems and projections; Data formats: ESRI shape file, Mapinfo tab, Autocad dwg, Google kml, ESRI geodatabase, etc. Software: comercials (proprietary) and open source, desktop/server/web/mobile. GIS applications: environment, cadastre, transportation, military applications, private sectror, etc.
GIS on the web
GIS on the web Web GIS: computer application to serving data over a network for browser usage in a form of web services and web maps. Geoportal: a type of web portal used to find and access geographic information (geospatial information) and associated geographic services (display, editing, analysis, etc.) via the Internet.
GIS on the web: metadata Metadata: is data that provides information about other data. Metadata can be created for geographic data, web services or web applications. Metadata are used to create a catalog of metadata for search and use data in distributed systems. Key metadata standards are ISO19119 (for web services) and ISO19115 (for data).
GIS on the web: open data Open data: the idea that some data should be freely available to everyone to use and republish as they wish, without restrictions from copyright, patents or other mechanisms of control. The goals of the open data movement are similar to those of other "open" movements such as open source, open hardware, open content, and open access.
GIS on the web: Standards Interoperability: is referring to the ability of diverse systems and organizations to work together (inter-operate). On a technical level interoperability defines common data models, protocols for exchange of data, machine readable formats for data re-use, sharing and exchange. Interoperability as achieved through the usage of standards.
GIS on the web: Standards Open Geospatial Consortium - OGC http://www.opengeospatial.org/ - international organization with over 500 members (companies, NGOs, universities, governmental agencies, etc.) which are working towards interoperable standards in Geo-spatial domain. OGC specifications for: WMS, WMTS, WFS, CSW, SOS, etc.
GIS on the web: Standards WMS - A Web Map Service (WMS) is a standard protocol for serving (over the Internet) georeferenced map images which a map server generates using data from a GIS database. The Open Geospatial Consortium developed the specification and first published it in 1999. WMTS - A Web Map Tile Service (WMTS) is a standard protocol for serving pre-rendered georeferenced map tiles over the Internet. The specification was developed and first published by the Open Geospatial Consortium in 2010.
GIS on the web: Standards WFS - Web Feature Service Interface Standard (WFS) provides an interface allowing requests for geographical features across the web using platform-independent calls. WFS-T - WFS - Transactional allows creation, delition and update of features.
GIS on the web: Standards WCS - The Open Geospatial Consortium Web Coverage Service Interface Standard (WCS) defines Web-based retrieval of coverages that is, digital geospatial information representing space/time-varying phenomena. SOS - The Sensor Observation Service (SOS) is a web service to query real-time sensor data and sensor data time series and is part of the Sensor Web.
GIS on the web: Standards CSW: Catalog Service for the Web is a standard for exposing a catalogue of geospatial records in XML on the Internet (over HTTP). The catalogue is made up of records that describe geospatial data (e.g. KML), geospatial services (e.g. WMS), and related resources. OGC Catalog Service, defines common interfaces to discover, browse, and query metadata about data, services, and other potential resources.
GIS on the web: SDI SDI: A spatial data infrastructure (SDI) is a data infrastructure implementing a framework of geographic data, metadata, users and tools that are interactively connected in order to use spatial data in an efficient and flexible way.
GIS on the web: SDI European agenda in SDI context: Directive INSPIRE (2007/2/EC), establishing establishing an Infrastructure for Spatial Information in the European Community (INSPIRE) and effective from 15th May 2007.
GIS on the web: INSPIRE INSPIRE (http://inspire.ec.europa.eu/): To ensure that the spatial data infrastructures of the Member States are compatible and usable in a Community and transboundary context, the Directive requires that common Implementing Rules (IR) are adopted in a number of specific areas (Metadata, Data Specifications, Network Services, Data and Service Sharing and Monitoring and Reporting).
GIS on the web: INSPIRE INSPIRE principles: Data should be collected only once and kept where it can be maintained most effectively. It should be possible to combine seamless spatial information from different sources across Europe and share it with many users and applications. It should be possible for information collected at one level/scale to be shared with all levels/scales; detailed for thorough investigations, general for strategic purposes.
GIS on the web: INSPIRE INSPIRE principles: Geographic information needed for good governance at all levels should be readily and transparently available. Easy to find what geographic information is available, how it can be used to meet a particular need, and under which conditions it can be acquired and used.
INSPIRE Annex 1: GIS on the web: INSPIRE
INSPIRE Annex 2: GIS on the web: INSPIRE
INSPIRE Annex 3: GIS on the web: INSPIRE
INSPIRE uses: GIS on the web: INSPIRE
GIS on the web: WISE WISE (http://water.europa.eu/): WISE is a partnership between the European Commission (DG Environment, Joint Research Centre and Eurostat) and the European Environment Agency.
GIS on the web: WISE The Water Data Centre, hosted at the European Environment Agency (EEA), provides a central access point to several web-services: interactive maps, data viewers, European datasets and indicators. These services are mostly based on reporting from countries as part of implementation of EU directives or via the Eionet framework.
GIS on the web: WISE WISE reporting template for floods:
GIS on the web: SEIS The Shared Environmental Information System is a collaborative initiative of the European Commission and the European Environment Agency (EEA) to establish together with the Member States an integrated and shared EU-wide environmental information system. This system would tie in better all existing data gathering and information flows related to EU environmental policies and legislation. It will be based on technologies such as the internet and satellite systems and thus make environmental information more readily available and easier to understand to policy makers and the public. http://ec.europa.eu/environment/archives/seis/what.htm
GIS on the web: ICPDR The International Commission for the Protection of the Danube River (ICPDR) works to ensure the sustainable and equitable use of waters and freshwater resources in the Danube River Basin. https://www.icpdr.org/. There are reporting GIS templates in *.shp files, with version 4.1.1 implemented for EPIRB GIS data and portal.
GIS on the web: ICPDR reporting templates
EPIRB system architecture: activities
EPIRB system architecture: activities 1. Data inventory elaborated review of over 1000 files, delivered for 6 countries was done during very early stage of the project. Detail information was provided in the inception report, were key issues with data were addressed. Information for the data condition of field data survey was also included. 2. Logical data model based on the data inventory and project requirements (ICPDR reporting templates) we have identified key data themes and developed the logical data model.
EPIRB system architecture: activities
EPIRB system architecture: activities General information: the data model was developed in ESRI Personal Geodatabase Format, used in all six countries, in common projection coordinate system (WGS 1984 UTM). Each country has two databases: Production database this database contain 9 data sets, organized logically. 1. Filled ICPDR Dataset: has the information for ICPDR data for each country. The number of layers varies, based on the data availability or unclear status. 2. Administrative data set contains layers with administrative data (e.g. borders, administrative units, settlements, etc.).
EPIRB system architecture: activities 3. GW contains data for groundwater (e.g. wells, groundwater bodies, etc.); 4. SW contains data for surface water (e.g. rivers, lakes, water bodies, etc.) 5. Infrastructure contains data for infrastructure (roads, mines, etc.); 6. Other contains data for uncategorized information. 7. Field protocols contains information Template database with ICPDR reporting data, containing all required templates (35 feature classes and 8 tables). This data can be used for further data development in the six countries once layers are collected.
EPIRB system architecture: activities 3. Physical data model the physical data model is developed following the logic and organization from the logical data model. All data sets are repeated for all basins. For further improvement of the data model we strongly recommend a standardization of the naming convention of the layers (e.g. English naming, CamelCase structure, singular naming, etc.). 4. Data loading and migration data loading was done by importing identified layer from step 1. 5. Map preparation map preparation was done for all layers, which the team manages to identify and load into the ICPDR templates.
EPIRB system architecture: technological stack Operating system: Ubuntu Linux Server 14.04 DBMS: Postgre SQL 9.3 / PostGIS 2.1 Web Server: Apache Tomcat 7 Application GIS server: Geoserver 2.5.1 (http://geoserver.org/) Geospatial SDI platform: Geonode 2.4 (http://geonode.org/) Metadata platform: PYCSW (http://pycsw.org/) Desktop GIS software: QuantumGIS ( http://www.qgis.org/ ) The system is physically located on a cloud based infrastructure on a https://www.leaseweb.com/ virtual server.
EPIRB system architecture: technological stack PostGRE / PostGIS: PostgreSQL is a powerful, open source object-relational database system. It has more than 15 years of active development and a proven architecture that has earned it a strong reputation for reliability, data integrity, and correctness. It runs on all major operating systems, including Linux, UNIX (AIX, BSD, HP-UX, SGI IRIX, Mac OS X, Solaris, Tru64), and Windows. PostGIS is a spatial database extender for PostgreSQL object-relational database.
EPIRB system architecture: technological stack Geoserver: GeoServer is an open source server for sharing geospatial data. Designed for interoperability, it publishes data from any major spatial data source using open standards.
EPIRB system architecture: technological stack Geonode: Open Source Geospatial Content Management System. GeoNode is a web-based application and platform for developing geospatial information systems (GIS) and for deploying spatial data infrastructures (SDI).
EPIRB system architecture: technological stack PyCSW: pycsw is an OGC CSW server implementation written in Python. Started in 2010 (more formally announced in 2011), pycsw allows for the publishing and discovery of geospatial metadata via numerous APIs (CSW 2/CSW 3, OpenSearch, OAI-PMH, SRU), providing a standards-based metadata and catalogue component of spatial data infrastructures. pycsw is Open Source, released under an MIT license, and runs on all major platforms (Windows, Linux, Mac OS X).
EPIRB system architecture: technological stack QGIS: The Leading Open Source Desktop GIS. QGIS is the best GIS tool in the free and open-source software (FOSS) community.
EPIRB system architecture: technological stack
EPIRB system architecture: organizational scope The system will include the following user types: 1. Power users/administrators: administrators for system management and data editors (power users) for data editing, queries, analysis, etc. 2. Users/specialists: users from different countries (project beneficiaries), EU institutions (e.g. JRC/ICPDR/EEA, etc.): access the database (in full) or web services (e.g. with user credentials or open access for view and download). 3. General public: open access to a web portal with functionality to view a web map.
EPIRB system architecture: informational scope
EPIRB system architecture: main functionality Desktop GIS Client: all functionality, supported by QGIS - data editing, data conversion, georefencing, digitizing, map preparation, spatial queries, etc. User can connect directly to the centralized database and have full access to all information (within their project areas/countries). Web GIS: Geoserver access to create and maintain web services (e.g. WMS/WFS), preview web services (e.g. through Open Layers).
EPIRB system architecture: main functionality Spatial content management system: Upload data (e.g. Shapefiles) for storage on the server and automatic creation of web services; Creation of web services and integration in web application; Publish and share web application on different level; Embed web applications in external sites; Metadata management (on web service level; on web application level); User management (groups and users); Documents management.
EPIRB system: key principles Build around open source technology that guarantees sustainability; Build according to international standards (OGC) and EU Directives (e.g. Inspire/WFD) that will allow integration with any EU or other information system (e.g. WISE/ICPDR/JRC/INSPIRE, etc.); Provide centralized repository for water related spatial data;
EPIRB system: key principles Standardize data, based on strict naming convention and standards; Functionality for data sharing and data exchange, based on open web services; Functionality for simple creation of web application (by non-specialist with minimum training);
EPIRB system: key principles Functionality for simple creation of web application (by non-specialist with minimum training); Establish a foundation for effective Spatial Data Infrastructure, which can be used as a good example in other application domains in the region; Set the standards for metadata preparation and usage (for data discovery and harvesting from other systems);
EPIRB system: key principles Avoid the creation of a black box as all the source code is available and free for use; Fill the gap between the technological expertise and decision making process/bodies. Establish a foundation for effective Spatial Data Infrastructure, which can be used as a good example in other application domains in the region;
EPIRB system: interface DEMONSTRATION: http://185.17.144.169/
EPIRB system: interface DEMONSTRATION: direct connection to the PostGIS DBMS:
EPIRB system: interface DEMONSTRATION: connection to the portal's WMS services using QGIS and ArcGIS. http://185.17.144.169:8080/geoserver/geonode/ows? Catalog service for the web, available at: http://185.17.144.169/catalogue/csw?service=csw
EPIRB Web GIS Portal Training Thank you! Lyubomir Filipov, GIS Consultant, e-mail: lubomirfilipov@gmail.com skype name: fipeto http://www.linkedin.com/in/lubomirfilipov