Continuous Spatial Data Warehousing

Transcription

1 Continuous Spatial Data Warehousing Taher Omran Ahmed Faculty of Science Aljabal Algharby University Azzentan - Libya Taher.ahmed@insa-lyon.fr Abstract Decision support systems are usually based on multidimensional structures which use the concept of hypercube. Dimensions are the axes of analysis and form a space where a fact is located by a set of coordinates at the intersections of members of dimensions. Conventional multidimensional structures deal with discrete facts linked to discrete dimensions. However, when dealing with natural continuous phenomena the discrete representation is not adequate. There is a need to integrate spatiotemporal continuity within multidimensional structures to enable analysis and exploration of continuous data. There is a multitude of research issues that lead to the integration of spatiotemporal continuity in multidimensional structures. In this paper, we discuss some of these, present briefly a multidimensional model for continuous field data. We also define new aggregation operations. The model and the associated operations and measures are validated by a prototype. 1. Introduction The last few years have seen an explosive growth of the size of data produced by the different kinds of sensors dispersed all over the surface of the earth. These sensors measure different kinds of natural phenomena at different time intervals. Exploiting these huge volumes of data in a decisional context is the main function of spatial data warehouses (SDW). Data warehouses are used in analysis purposes that involve examining data and possibly identifying relationships that may exist between different elements. Since we deal with information on or above the surface of the earth, therefore spatial positions are involved i.e. we deal with spatial data. In an operational processing, geographic information systems (GIS) have shown their ability in managing geographic data. The effectiveness of GIS comes mainly from their capacity of linking different information in a spatial context and in drawing conclusions from the different relations that exist between different phenomena. However GIS are oriented towards spatial data management not towards effective analysis. In this paper we present our research towards integrating spatiotemporal continuity in decision support systems. In the second section we discuss the differences between GIS and decisions support systems. Section 3 presents the research objectives, issues and motivations. The fourth section contains a theoretical background. In section 5 we go briefly over a multidimensional model for field based data and present a prototype of a continuous spatial data warehouse. The conclusions and future work are presented in section GIS and Decision Support Systems GIS is defined as an information system that is used to input, store, retrieve, manipulate, analyze, and output geographically referenced data or geospatial data [6]. The main capabilities of GIS are to integrate large volumes of spatial and non-spatial data and enhance problem understanding through data visualization in map forms. Multidimensional modeling and On-Line Analytical Processing (OLAP) allow intuitive, easy and fast analysis of huge volumes of data [8]. In general, OLAP provides aggregated summary information, collected from multiple sources, and modeled in multidimensional structures. Dimensions usually have hierarchical structures and correspond to a business perspective where each cell at the intersection of members of dimensions may contain the value(s) of some aggregated measure. Data hypercube refer to the computation of an aggregation over all combinations of dimensions.

2 OLAP technology on which decision support systems are based, involves entirely different concepts from the transactional architecture of GIS. This architecture is insufficient for knowledge discovery and data mining [3]. The concepts of dimensions, measures, hierarchies etc are not supported by GIS. Conversely, DSS do not support cartographic display which can be helpful for decision making. Spatial decision support systems are designed to provide analysis tools and help decision makers solve complex spatial problems. Providing a dependable spatial decision support requires the coupling of GIS and OLAP so that the latter provides multidimensionality and the first manipulates spatial data [10]. 3. Motivations and Research Issues Based on the nature of data represented there two data conceptualization in GIS. The first is discrete which concerns spatial data that have clear boundaries and the second is continuous which deals with natural phenomena that exist continuously in time and in space. Despite the huge amounts of continuous data collected by all types of sensors there has not been much work done on data warehouse modeling for this type of data. Most of the work on SDW and SOLAP deal with discrete spatial data. To the best of our knowledge the work done on spatial decision support systems deals only with the discrete representation. Even when continuous data are dealt with in a decisional context [7][12][14] they are treated as discrete. In conventional multidimensional structures all data involved are discrete. Dimensions are organized in discrete hierarchical levels where each level has a finite set of discrete members. A discrete fact may be found at the intersection of members. However for continuous data the spatial and temporal dimensions are not discrete and must not be treated as such. Phenomena take place everywhere and constantly without disruption. Nevertheless, they cannot be measured continuously at all points in space nor can these measurements be stored in databases due to several factors like the discrete nature of computers. Only samples are measured and stored which yields a discrete representation of a continuous phenomenon. This representation is reflected in discrete spatial and temporal dimensions in SDW and SOLAP. Since a distinction is made in GIS on the detailed data level with respect to the continuous and discrete representation, the same distinction should be made on the aggregated level. Our main objective is integrating continuity within multidimensional structures to enable analysis and exploration on both continuous and discrete data. Integrating spatiotemporal continuity within multidimensional structures accomplishes the following objectives [1]: - Recover hidden information: A continuous representation can recover lost or hidden information that results from representing continuous data discretely. - Analysis at Detailed Levels of Hierarchy: There are numerous cases where the need of low granularity analysis arises as in the cases of disaster management. This type of analysis is can be made possible by continuous multidimensional structures. - Continuous analysis: Because of the discrete representation, natural phenomena are analyzed as discrete spatial objects and as snapshots of data values over different periods of time. It would be more realistic to analyze natural phenomena as they occur in real life where they evolve continuously in space and in time rather than as a collection of discrete pieces. There are several issues involved in integrating continuity in multidimensional structures [1]: - Continuous multidimensional model. All existing models deal with discrete dimensions related to discrete facts. To the best of our knowledge, no work has been done on modeling multidimensional structures for continuous data with the exception of [12] who focuses on using the known density of data to calculate queries without accessing the data. - Range of continuity. One of the important issues is where does the continuity begin and end hierarchically? Does continuity imply producing new finer levels or should it only produce data for any instant in time? And can higher levels be continuous? - Choice of Interpolation methods. Interpolation methods differ in their assumptions, methodologies, complexity, and deterministic or stochastic nature. In addition the performance of the method is an important factor for DSS since they require fast response. - Storage and optimization. To obtain fast response OLAP uses pre-aggregation to eliminate the overhead of calculating SQL aggregations during run time. For continuous data, the complexity of the problem is augmented because of interpolation. Interpolating means that pre aggregation can not be performed for several or all operators.

3 - Operations in continuous hypercubes. The classic OLAP operators need to be extended to be applicable in the new structures. The introduction of continuity could change the results of different operations. Also there will be need to formally describe new operators. As an example, the operation sum in the discrete structure will be integration in the continuous structure. - Result visualization. Results must be displayed in an intuitive and attractive manner to provide a user friendly environment for decision making. This includes cartographic display, grid, graphic representation etc. It is also crucial to give an indication of how good the estimation is through the use of quality indicators. 4. Theoretical Background Despite the multitude multidimensional models proposed in the literature, no consensus has been made on one formal model. In spatial data warehousing, most of the work focuses on case studies and prototypes with little concentration on sound formalism. In [9] a framework for multidimensional exploratory spatiotemporal analysis is proposed. It focuses mainly on the hierarchy theory. A dimension hierarchy is defined as a 4-tuple H = (V, F, G, ) where V denotes the node or the vertex of a hierarchy. Each member of V is associated with a domain of elements. G is the dimension path which is a totally ordered list of nodes. The symbol denotes that a dimension path is a linear totally ordered list of nodes. Every adjacent vertex pair in a path is associated with a partition mapping function called, categorization function, F = {f 0, f 1, f 2, } such that fi = f i : domain (v i ) domain (v i+1 ) i 1. The main drawback is the lack of formalism of the common operators. [11] extends a conceptual multidimensional model with spatial dimensions, spatial hierarchies and spatial measures. A multidimensional model is defined as a finite set of dimensions and fact relationships. A dimension is composed of hierarchies which are composed of one or several levels represented as entity types. Hierarchies that contain only one level are called basic hierarchies. Levels are related to each other by a partial order relationship. For any two consecutive related levels l i, l j, the level l j is called parent and l i is called child if l i l j. They also define a category attribute that shows how child members are grouped. In [4] a model that introduces the concept of entity schema and entity instances is presented. The entity schema S e is a tuple of attributes a i defined on a domain dom(a i ) where an attribute is an alphanumeric identifier. The entity instance t i over S e is a tuple linking values from dom(a i ) to each attribute a i. Entity schemas and instances are used to model real world objects of a multidimensional application model. Fact tables are represented by base cube schemas where all dimensions are at their lowest level except the measure which can be at any level. A base cube schema SBC bc is tuple SBC bc = S 1, S m, S f, δ where no 2 schemas are equivalent and δ is a Boolean function indicating whether a value exists at any combination of instances. Cubes are base cubes after aggregation. All of these models deal only with discrete spatial data. 5. A Continuous MD Model The main objective of our work is to define a new model or extend an existing one so that it takes into account spatiotemporal data characteristics. The model proposed in [15] is the best candidate for extension since it formally defines most of the OLAP operations and it is based on the idea of basic cubes that allows serial performing of operations. In this paper we present some of the necessary elements we added. More details are found in [1]. 5.1 Basic cubes We distinguish two types of basic cubes. The first type of basic cubes is the discrete basic cube discc b which is a 3-tuple <D b, L b, R b > where D b is a list of dimensions including a dimension measure M. L b is the list of the lowest levels of each dimension and R b is a set of cells data represented as a set of tuples containing both level members and measures in the form of S=[s 1, s 2, s 3, s n,m] where m is the dimension that represents the measure. To achieve a continuous representation of the basic cube, estimated values are derived from the discc b. Estimated measures related to the infinite members of a given continuous dimension are estimated using actual cell values from the discc b. This is achieved by applying interpolation functions to a sample of the discc b values which will give the second type of basic cubes "continuous basic cube" contc b. The number of tuples in the contc b is theoretically infinite since a dimension level in the class of continuous dimensions contains an infinite number of members. We define

4 contc b, as 4-tuple <D b,( D b, F), L b, R b >. Where D b, is a set of discrete dimensions and D b is a set of continuous dimensions. It is clear seen that discc b contc b as sample values are included within the contc b. 5.2 Cubes Cubes are built from basic cubes. A cube C is defined as 4-tuple <D, L, contc b, R> where D is a list of dimensions including M as defined above, L is the respective dimension level, R is cell data and contc b is the basic cube from which the cube C is built. Because of the nature of continuous field data, different aggregation functions are used to build the cube at higher dimension hierarchies. For example, the sum of the measure for a specific region will be represented as the integration of the function representing the phenomenon. Other aggregation functions like min, max or average will be performed on contc b and their results will be assigned to higher levels of the hierarchy. Based on data values used to obtain an aggregation, the aggregation on continuous multidimensional structures can be classified as either discrete or continuous. Discrete operations use only sample data values and their aggregations will correspond to the discrete higher levels. Continuous operations use all data values of the field (sample and estimated values). Their aggregations correspond to the higher levels resulting in aggregated values based on continuous representation. The same multidimensional schema is used for both classes with the only difference being the detailed data used in calculating the aggregations. One can imagine the existence of a parallel hierarchy that is used for the continuous representations (Figure 1). It should be noted that continuous aggregated values and discrete aggregated values are not necessarily equal. Figure 1. Continuous and discrete representation of dimensions Discrete aggregations : They include the conventional OLAP operations and are calculated based on the real observed values. We list here the most common operations: DiscMax = v i such that v i > v j V* DiscMin = v i such that v i < v j V* DiscSum = n v i i = 1 of observed values. n v i for v i V* where n is the number DiscAvg = i for v i V* where n is the Card (V *) number of observed values. Continuous aggregations The second category of operations concerns the operations that involve all domain values of the phenomenon i.e. all observed and estimated values of the phenomenon are used to produce a hypercube based on the continuous representation. In addition to aggregation functions a new calculated measure has been defined that apply only on continuous data. Continuity is considered on either an interval of time or on a specified region. The continuous aggregations on the measure of the continuous field are 1 : ContMax = v i such that v i > v j V ContMin = v i such that v i < v j V ContSptSum = f x, y, t ) dxdy 1 Definition of continuous field from [1]. ( where (x, y) D

5 ContTmpSum= f ( x, y, t) dxdydt where (x,y) D ContSptAvg= ) ContTemporalAvg = f ( x, y, t dxdy where (x, y) D area f ( x, y, t ) dxdydt where t t 2 1 [t 1 : t 2 ] is an interval of time and (x, y) D The calculated measure gradient, which is the change of the value of the field that results by a change of a unit of space, can be applied to most detailed level of data and its results can be aggregated to higher levels : Gradient = grad((x, y), t) = v= (x, y) D. f f f,, x y t where 5.3 Continuous SOLAP prototype One of the applications that can benefit from continuous multidimensional structures is air pollution analysis. We designed and implemented a prototype of an application that observes air pollution in order to validate the model defined above and to show the potentials of what we termed Continuous SOLAP or CSOLAP. In order to validate our model and test the performance of our data warehouse a considerable volume of data is required. We used data published by AIRPARIF [2]. However this set covers only the Parisian region so we had to simulate data values for the rest of France and we ended up with a large set of data that we stored in an SQL Server data warehouse. To create a continuous representation all dimensions must be at their lowest level then we choose an interpolation function from a list of different functions. This will create a thematic map on the fly. The thematic map contains all values of the field for the chosen dimensions (Figure 2). Figure 2. Interface of CSOLAP We can navigate on this map or from the menus and tools bars to obtain higher level aggregations. 6. Conclusions Spatial decision support systems have benefited from advances made in OLAP and data warehouses technologies. The emergence of SOLAP has made spatial decision support easy and flexible. However the perception of space and time is still limited to the discrete perception which does not represent natural phenomena correctly. Our work aims at the integration of continuity in multidimensional structures. Starting from a discrete hypercube, a continuous hypercube is created by applying interpolation functions over cell data. We have two representations: the first is an internal representation, the second is an external representation. A multidimensional model dedicated to continuous field data was defined along with a set of operations. The model and the prototype are validated by a prototype of a data warehouse of air pollution. However not all research issues listed in this paper have been looked into. Our main objective was defining a formal model. Defining additional operators, metadata management, dealing with different intervals of time, storing continuous hypercube and ad-hoc hierarchies are a list of the perspectives of this research. 7. References [1] T. O. Ahmed, and M. Miquel. Multidimensional Structures Dedicated to Continuous Spatiotemporal Phenomena. Proc 22 nd British National Conf. on Databases (BNCOD22), Sunderland. 2005, [2] AIRAPRIF, Monitoring the quality of air in Ile de France. [3] Y. Bédard, T., Merrett, and J. Han. Fundamentals of Spatial Data Warehousing for Geographic Knowledge Discovery in Geographic Data Mining and Knowledge Discovery. Research Monographs in GIS series edited by Peter Fisher and Jonathan Raper [4] Bimonte, S., Tchounikine, A. and Miquel M. Towards a Spatial Multidimensional Model. In the Proc. of the ACM 8 th Int. Workshop on Data Warehousing and OLAP, 31 Oct. - 5 Nov. 2005, Bremen, Germany. [5] Cowen, D. J. GIS versus CAD versus DBMS: What are the differences? Fotogrammetric Engineering and Remote Sensing, 54, 1988, [6] M. Goodchild. Geographical information science. International Journal of GIS, 2003, 6, [7] H. Hasenauer, I. Haslik, R. Rosenthaler, G. Pernul and D. Stangl. Conceptual framework of a data warehouse for the National park Hohe Tauern. Proc. 13 th Int. Symposium "Informatik für den Umweltschutz" der Gesellschaft für Informatik (GI), Magdeburg, 1999,

6 [8] W. H. Inmon. Building the Data Warehouse. John Wiley and sons [9] Kemp, Z. and Lee, H Multidimensional Model for Explatory Spatiotemporal Analysis. Proc. of the 5th International Conference on GeoComputation, University of Greenwich, UK. [10] Z. Kouba, K. Matousek and P. Milkovsky. On Data Warehouse and GIS integration. Proc. of the 11 th Int. Conf. and Workshop on Database and Expert Systems Applications, Greenwich, 2000, [11] E. Malinowski and E. Zimányi, Representing Spatiality in a Conceptual Multidimensional Model. GIS 04. Washington, DC, USA pp [12] D. G. Morgan, and T. Glover. Distributing Data Ownership: The Northwestern Geospatial Data Network. GIS Vancouver, B.C., February [13] J. Shanmugasundaram Fayyad, U. M. and Bradely, P. S. Compressed data cubes for OLAP Aggregate Query Approximation on Continuous Dimensions. Proc. of the 5 th ACM SIGKGG Int. Conf. on Discovery and Data Mining (KDD99), New York, 1999, [14] X. Tan. Data Warehousing and Its Potential Using in Weather Forecast. Proc. 22 nd Int. Conf. on Interactive Information Processing Systems for Meteorology, Oceanography, and Hydrology. Atlanta, GA [15] P. Vassiliadis. Modeling Multidimensional Databases, Cubes and Cube Operations. Proc. of the 10 th Int. Conf. on Scientific and Statistical Database Management (SSDBM), Capri, Italy, 1998.