Vector tiles for the Swiss Federal Geoportal Ingensand Jens, Marion Nappez1, Cédric Moullet, Loïc Gasser2,Sarah Composto1 1 GIS Lab, University of Applied Sciences Western Switzerland. Route de Cheseaux 1, CH-1401 Yverdon-les-Bains {jens.ingensand, marion.nappez, sarah.composto}@heig-vd.ch 2 Swiss Federal Office of Topography. Seftigenstrasse 264, CH-3084 Wabern {cedric.moullet, loic.gasser}@swisstopo.ch Abstract Vector tiling aims at cutting vector data into smaller entities. It has several advantages for the development of web-mapping systems, such as the possibilities to apply different styles, to access attributes or to render 3D data. In this paper we investigate several parameters that need to be considered for the generation of vector tiles. Furthermore we present several options that will be tested in order to establish vector tile services for future versions of the Swiss Federal Geoportal. Keywords Vector Tiling, Web GIS, WebMapping, Web Services, Geoportal, Vector Data 1 Introduction The principle of vector tiling is similar to raster data tiling where large raster data sets are tiled into smaller pieces and stored in hierarchical structures, either in databases or in file systems. Raster tiling allows for an efficient consumption of raster data through a network connection, for instance using the standardized protocol WMTS (OpenGIS Web Map Tile Service, www.opengeospatial.org) implementation standard. Within the context of web-mapping systems vector data, as compared to raster data, has several advantages first, vector data allows for more flexibility for rendering maps, such as the possibility to apply different styles or rendering 3D data. Second, it is possible to transfer not only geometries, but also an entities attributes. A third advantage is that, depending on the data layer, the features' attributes, the feature density and the level of detail, data can be compressed at a higher level. A fourth advantage is the fact that data can be reused and transformed on the client side; for instance using coordinate transformation, spatial analysis, and so forth. However transferring vector data from a server also bears certain risks; e.g. the fact that data can be illegally downloaded and reused for other uses that were not intended by the administrators of a server infrastructure. 717
The idea of tiled vector data services is to combine both the advantages of vector data with the advantages of tiled raster data services. Figure 1: Screenshot of the Swiss Federal Geoportal map.geo.admin.ch Today the Swiss Federal Geoportal map.geo.admin.ch is the official geoportal of the Swiss state, serving almost 400 data layers (see Figure 1) (Moullet 2015). Map.geo.admin.ch is based on the open-source framework MapFish (mapfish.org) which itself consists of several Python and Javascript libraries such as Pylons (www.pylonsproject.org), OpenLayers (www.openlayers.org), ExtJS (www.sencha.com/products/extjs) and GeoExt (geoext.org). The geoportal uses Amazons EC2 and S3 (Amazon 2015) cloud computing infrastructure as a service provider. A majority of the available data layers are generated using WMTS web services; some layers are available as WMS (Web Map Service, www.opengeospatial.org) services. About 2'500'000'000 WMTS unique raster tiles are stored in the cloud infrastructure (Oesch 2014). One problem in the context of vector tiling is the absence of standards. There are standards for the transfer of vector data such as WFS (Web Feature Service, www.opengeospatial.org) and standards for serving tiled raster data such as WMTS, but today there is no standard for tiled vector data. Possible reasons for this lack of standards might be found in the complexity of vector tiling. Within the context of the Swiss Federal Geoportal, the goals of the project GeoTile are: to investigate existing vector tile service solutions and to analyze which are the different parameters that need to be taken into account for the implementation of tiled vector services and for the development of 718
clients that need to assemble data layers from cut vector data to analyze which of the identified parameters and existing solutions would be usable for the Swiss Federal Geoportal to test different parameters in order to identify the best solution for tiled vector services In the following sections we will illustrate different solutions and concepts for generating, serving and consuming vector tiles. Due to the vast scope of vector tiling, only the most important approaches are cited. Thereafter we will discuss which of these solutions will be tested in the Swiss Federal Geoportal infrastructure in order to identify the solution that is best adapted for future versions of the geoportal. 2 Different approaches to vector tiling 2.1 Regular versus irregular tiling A first question in the context of vector tiling is whether a vector layer should be cut into pieces of regular size (in terms of equally-sized and shaped bounding-boxes, see Figure 2), whether a vector layer should be cut into pieces with a similar payload or if each object (such a polygon or a point) should be stored in a tile of its own (see Figure 2) Figure 2: Examples for irregular and regular vector tiling. Irregular tiling: each object is a tile of its own (left) Regular tiling: a square bounding-box is applied to cut a layer into equallysized and shaped tiles (right) Today several applications such as Google Maps (maps.google.com, Google Inc, Mountain View (CA), USA), GISCloud (www.giscloud.com, London, UK) or Polymaps (polymaps.org) have adopted the concept of regular vector tiling. One notable example for irregular tiles has been described by Dufilie and Grinstein (2014). In their approach shapes are cut using the TileSplit algorithm into pieces of equal payload. The results are both an overview layer that indicates the positions of the tiles and the tiles themselves. In order to generate tiles for different levels of detail, the authors used Visvalingam's algorithm (Visvlingam and Whyatt 1992). 719
2.2 Formats for storing and transferring vector tiles Similar to raster tiles, vector tiles can be stored in database structures or file systems. Two common formats for transferring vector data in a web context are XML-based formats such as GML and JSON-based formats such as GeoJSON and TopoJSON. Both types of formats have advantages and disadvantages. XMLbased formats are extensible and offer several possibilities to verify data. The disadvantage of XML-based formats is the fact that much storage is used for tags. JSON files on the other hand need less space and JSON files are easily parsed using Javascript (JSON 2015) 2.3 Generalization and simplification of vector data Depending on the zoom level, data needs to be simplified and generalized in order to maintain readability and payload. Several methods and algorithms could be used within the context of tiled vector data. One possibility to generalize vector data is the GAP (Generalized Area Partitioning) tree method which focuses on topological area data. This method identifies the importance of areas. Depending on the level of detail less important areas are removed (Van Oosterom 1993). Tinghua and Van Oosterom 2002 have extended this method in order to include bridge areas between aggregated polygons. Another algorithm described by Visvlingam and Whyatt (2002) focuses on the importance of a line's edges. In this method an edges neighbors are used to calculate the area of a triangle. Depending of the calculated area, an edge is considered more or less important. If the line needs to be simplified, less important edges are removed. This method can be used both for linear features as well as for area features. The Douglas-Peucker (Douglas and Peucker1973) algorithm is perhaps the most widely used algorithm for generalization in GIS software. The algorithm uses distances between the points of a line to determine which points can be omitted. For point features, there are basically two possibilities to reduce the number of points: Either points can be grouped into clusters, for instance using the KMeans clustering algorithm (MacQueen 1967), or thinned such as described by for instance Moenning and Dodgson (2003). In both cases the loss of information (e.g. in terms of attributes) is an important subject to consider. 2.4 Attribute handling If geometric entities are cut into several pieces, one important question is how attributes are handled. One solution to this problem is to replicate the attributes to all pieces of the cut vector feature and thus to increase the overall payload. Another possibility is to attach the attributes to one of the pieces and to enable clients to reassemble geometries and attributes. A third solution is to transfer attribute data separately and independently from the geometries. 720
2.5 Client-side object assembly A challenge in the context of vector tiling is the assembly of cut entities into the original vector features. The main question is how a client can know that all pieces of one entity have been transferred. The simplest method is to check all tiles that have been received for e.g. the existence of pieces of one specific entity. The drawback of this method is the time that is necessary to analyze all tiles. Another method has been described by Nordan (2012) in order to locate pieces of tiled vector features, pointers can be included in the pieces, indicating in which directions a client needs to look for more pieces. 3 Identification of the best approach In order to find the method that is best adapted for the Swiss Federal Geoportal, a testbed has been deployed. This testbed includes a copy of several data layers stored in PostgreSQL/PostGIS databases. Python scripts will be used to connect to these data sources and to produce vector tiles. The following options have been chosen: Only tiles with equally shaped bounding-boxes will be generated (regular tiling). The reasons for this decision are that the effort to generate irregularly shaped tiles appears to be greater than for generating regularly shaped tiles; second OpenLayers, the Javascript library that is used in the geoportal already has a built in support for regularly shaped raster-tiles (WMTS). Thereby an extension of this library (in order to enable the consumption of vector tiles) would be easier to accomplish. Furthermore an idea is to use the WMTS tile-indexation for the generation of the tiles (e.g. a vector tile would be addressed using a similar URL as a raster tile) to simplify the assembly of both raster- and vector tiles in a client. Several algorithms for the generalization of vector tiles will be tested. All tiles at all levels of detail will be generated at once. Thereafter we will analyze the payload, the stored features, etc. with statistical methods. Only JSON-based formats will be considered. The main reason is that JSON-based formats are lighter than XML-based formats in terms of payload and data processing. Attributes will be handled in separate files, independently from the vector tiles. This has the advantage that no redundant data will be generated. The main objective of our approach is to find a solution that balances flexibility, performance and complexity. Important parameters that will be considered are the payload of the generated tiles as well as the time it takes to generate the tiles for a given layer. Moreover the generated tiles will be used as a repository for the development of the Javascript client. 4 Discussion and conclusions Vector tiling paired with webgl-enabled Javascript libraries permits to create high-performance web-mapping applications. Today mostly proprietary solutions such as Google Maps are using vector tiles. We can state that there is a lack of standards for the generation and consumption of vector tiles. Vector tiling however is a complex subject as there are many different possibilities to 721
implement such services depending on the context of the application. In Google Maps for instance there is a limited number of data layers available, however these data layers cover the surface of the entire planet. The Swiss Geoportal on the other hand offers hundreds of data layers from point feature layers containing all addresses in the whole of Switzerland to polygon layers with all of Switzerland's buildings. Our approach to tiled vector data refers to existing standards such as the WMTS tile indexing and JSON-based file formats. Moreover we will use a testbed that enables empirical testing of different alternatives. Since this project is an ongoing project, at the time of writing no empirical results are available. References Amazon (2015): AWS Case Study: Swisstopo. Amazon web services portal: http://aws.amazon.com/fr/solutions/case-studies/swisstopo/ (9.3.2015) Douglas, D., Peucker T. (1973): Algorithms for the reduction of the number of points required to represent a digitized line or its caricature. In: The Canadian Cartographer. (10) 2. 1973, pages 112 122. Dufilie and Grinstein (2014): Feathered Tiles with Uniform Payload Size for Progressive Transmission of Vector Data. In: Pfoser and Li (eds): Web and Wireless Geographical Information Systems. 13th International Symposium, W2GIS 2014, Seoul, South Korea, May 29-30, 2014. Proceedings. Pages 19-35 Springer JSON (2015): JSON: The Fat-Free http://www.json.org/xml.html (9.3.2015) Alternative to XML. MacQueen, J.B. (1967): Some Methods for classification and Analysis of Multivariate Observations, Proceedings of 5th Berkeley Symposium on Mathematical Statistics and Probability, Berkeley, University of California Press, 1:281-297 Moenning, C. and Dodgson, N.A (2003) A new point cloud simplification algorithm. Proceedings of the 3d IASTED International Conference on Visualization, Imaging and Image Processing (VIIP 2003), 8-10 September 2003, Benalmádena, Spain Moullet, C. (2015): Géoportail de la confédération map.geo.admin.ch: contenu et évolution technologique. Journée de veille technologique IGSO, 28.1.2015, Yverdon-les-Bains, Suisse Nordan, R.P. (2012) An Investigation of Potential Methods for Topology Preservation in Interactive Vector Tile Map Applications. Trondheim University, Norway. 722
Oesch, D. (2014): The Swiss Federal Geoportal www.geo.admin.ch - more than maps. Geospatial World Forum 2014 8.5.2014 Geneva. Tinghua, A. and Van Oosterom, P (2002): GAP-Tree Extensions Based on Skeletons. In: Proceedings of the 10 th International Symposium on Spatial Data Handling. July 9-12 2002, Ottawa, Canada Van Oosterom, P (1993). The GAP-tree, an approach to On-the-Fly Map generalization of an Area Partitioning. In: Proceedings of the GISDATA Specialist Meeting of Generalization. Compiègne, France, 15-19 December 1993 Visvalingam, M. and Whyatt, J.D. (1992): Line Generalisation by Repeated Elimination of the Smalles Area. Cartographic Journal., 30 (1) pages 46-51 723
724