Index Selection Techniques in Data Warehouse Systems

Transcription

1 Index Selection Techniques in Data Warehouse Systems Aliaksei Holubeu as a part of a Seminar Databases and Data Warehouses. Implementation and usage. Konstanz, June 3, 2005

2 2 Contents 1 DATA WAREHOUSES 1 Data warehouses Data warehouse design Logical design Physical design Architectural sketch 4 3 Plan generation Execution plan operators Selection of the execution plan The cost model The index selection algorithm 8 5 Experimental results 9 6 Related work 10 6 Summary 10 Literature 10 A Index selection Algorithm in pseudo code 11 B Cost functions 12 Introduction The goal of this elaboration is a brief overview of the certain phases and techniques in a data warehouse (DW) designing process, aimed at optimizing the query processing. Basic issue is given to the physical design. Execution Paths Generator tool and a Cost Evaluation model are to be introduced as the important components, and, finally, a greedy algorithm is presented, which selects an optimal index set to be built in a DW implemented on a RDBMS respecting a constraint on the disk space devoted to indexing. 1 Data warehouse A data warehouse [1] is a collection of data from multiple sources, integrated into a common repository and extended by summary information (such as aggregate views) that is used primarily in organizational decision making.

3 3 1.1 Data warehouse design 1 DATA WAREHOUSES Data warehouse design methods consider the read-oriented character of warehouse data and enable the efficient query processing over huge amounts of data. DW design includes logical and physical phases aimed at improving the system performance. On the logical level the so-called view materialization[2], which strongly impacts the performance, takes place. However indexing techniques, which along with all the issues related to implementing the DW on a specific DBMS considered by physical design, have the fundamental meaning Logical design The multidimensional view of data in the DWs implemented on relational DBMSs is achieved by adopting different schemes. A special type of relational database schemas, called star schema (Picture 1), is often used to model the multiple dimensions of warehouse data (in contrast to the two-dimensional representation of normal relational schemas). In this case, the database consists of a central fact table (FT) and several dimension tables (DT). The FT contains tuples that represent business facts (measures) to be analyzed, e.g., sales or shipments. Each fact table tuple references multiple dimensional table tuples each one representing a dimension of interest like products, customers, time, region or salesperson. Dimensions usually have associated with them hierarchies that specify aggregation levels and hence granularity of viewing data. Since DTs are not normalized, joining the FT with the DTs provides different views (dimensions) of the warehouse data in an efficient way. Picture 1

4 4 2 ARCHITECTURAL SKETCH One of the most effective ways to minimize query response time during the logical design is view materialization. The underlining algorithm, driven by a workload (set of possible queries to be submitted to the system during operation), selects an optimal set of materialized views. Each view contains aggregated data obtained from the base fact table (FT) that includes elemental data; the aggregation level characterizing a view consists of attributes of the dimensional table (DT) Physical design Availability of the logical scheme enables further optimizations within the physical design phase. The set of indexes to be built on both FTs and DTs defined in this phase is another very important tool for speeding up the query response times. One of the possible ways to achieve this is to implement an optimizer model capable of determining an execution plan for each query and then let the greedy algorithm choose the most beneficial indexes with respect to the space constraints. Unlike the previous phases, physical design strongly depends on the features of specific DBMS: the categories of indexes available, the types of execution plans generated, the statistics consulted by the optimizer. 2 Architectural sketch M. Golfarelli, S. Rizzi and E. Saltarelli (DEIS University of Bologna) suggested the optimization method, which functional architecture can be represented by the following sketch (Figure 1). The approach features different components, which, given some input (ex. DW logical scheme, Workload, Data volume, System constraints etc.), according to the function they are responsible for, return some output (ex. Bound Workload, Indexable Attributes, Candidate Indexes, and Optimal Indexes) which either act as input for other component(s) or is returned as the final solution. The components involved in processing carry the following functions out: Aggregate Navigator: given a workload and a logical scheme including one or more materialized views, this component is in charge of selecting the best view on which each query should be

5 5 2 ARCHITECTURAL SKETCH solved. The aggregate navigator does not usually take indexes into account. Indexable Attributes Selector: based on the structure of the queries, this component determines which attributes of DTs could be usefully indexed. Candidate Indexes Selector: for each indexable attribute, this component evaluates which type of index is the most convenient. The indexes selected by this component, defined by couples (attribute; index type), are called candidate indexes. Optimal Indexes Selector: this component implements the algorithm which selects the indexes to be created. The optimal index set includes a subset of candidate indexes on DT attributes as well as all the indexes built on primary keys of DTs and FTs. Cost Evaluator: it is necessary to both the Generator of Candidate Indexes and the Optimal Set Generator to evaluate the access cost for each index. Plan Generator: given a physical scheme, a query and the FT on which it should be solved, it returns the best execution plan which solves the query. Figure 1

6 6 3 PLAN GENERATION 3 Plan generation As mentioned before, several components have to be implemented in order to perform the optimal indexing. One of them is a rule-based [3] optimizer that estimates the best execution plan for each query according to the view on which it will be solved. 3.1 Execution plan A query execution plan is a sequence of elementary operators applied to the physical scheme. Each operator (Table scan, Index scan, Table access, Index access, Hash join, Tid intersection) models a function carried out by the DBMS on either tables or indexes, which, if a local predicate specification is allowed, can provide additional filtration of the output. 3.2 Selection of the best execution plan The decision for selecting an execution plan for the query q is determined mostly by the number of the DTs on which at least one condition is expressed, which are called conditioned dimensioned tables (DTc): no DTc present FT is sequentially scanned then joined with all the DT involved in q through a nested-loop on their primary key index exactly one DTc present algorithm checks if there is an index allowing to access the FT from its foreign key referencing DTc and in this case for each tuple of DTc that satisfies condition access this index and the FT. Otherwise, a hybridhash join between DTc and the FT is executed. The result is joined with the other DTs requested in output. two or more DTc available for each algorithm decides how to carry out a join with the FT. Received tid sets obtained from different DTs are intersected then and the resulting tuples of the FT are accessed.

7 7 3 PLAN GENERATION The number of DT attributes on which a filter is defined and an index is built, α, drives the choice of the plan as follows: 1. if α=0, a sequential scan of the DT is executed, applying the filter to each tuple. 2. α=1 means that an index on a conditioned attribute is built. In this case an index scan is executed and, for each tid retrieved, the DT is accessed. 3. if α 2, all indexes on conditioned attributes are accessed, the tid sets obtained are intersected, possibly further filters are applied, and finally DT is accessed. Thereby, by means of a set of heuristic rules based on the database structure, the optimizer produces the execution plan without taking statistics to account. But the chosen physical scheme is valid also for cost-based DBMSs, since the statistics on data is used to further evaluate the cost of execution plan. 3.3 The cost model The cost model is adopted in order to compare different physical schemes and evaluated as a number of logical pages needed to be read to execute a plan. The cost function assigned to each operator in execution plan (Appendix B) except the aggregation, which is assumed to have a null cost. The cost of the full plan is evaluated as the sum of the costs for all the operators in it. Thus, according to the information required by the cost model the total cost for each execution plan can be calculated.

8 8 4 The index selection algorithm, 4 INDEX SELECTION ALGORITHM Due to its high complexity, the index selection problem is usually faced heuristically. As already stated, the view used to solve a query is selected during logical design, which is carried out neglecting the issues related to the indexing. Indexable attributes and primary keys of tables are the only elements that may be indexed. It should be noted that indexing an indexable attribute does not necessary lead to any performance improvement; on the other hand, once an index on an indexable attribute is built, the execution plans for all the queries in its support will use that index. Indexes in the physical scheme are independent of each other, but their contribution to the query execution cost depends on the table they are built on. The index selection algorithm (Appendix A) can be subdivided into three distinct sections. 1. Initialization of the set of candidate and optimal indexes as well as the available space for indexes on attributes other than primary keys. 2. Is determined in the while loop and carries out a greedy selection of indexes from the set C of candidate indexes for the workload based on the benefit per index page. If after inserting a new index into set O of the optimal indexes, it turns out that all the prime attributes of the fact table are indexed, one of these indexes must be transformed into the multiple-attribute index on the FT primary key; the choice is driven by the decay per index page related to the transformation. 3. Sets up the primary key indexes for the remaining FTs. If, for a given FT, a non-empty set of candidate indexes still exists, the one whose insertion in O as a multiple-attribute index on the primary key is cheapest is chosen. Otherwise, a non-indexable attribute is randomly chosen to build the multiple index.

9 9 5 Experimental results 5 EXPERIMENTAL RESULTS The approach has been tested on the TPC-H benchmark; 20 GPSJ queries inspired to those in the benchmark have been executed varying both their selectivity and the storage available for materialized views and indexes. The results proved that Indexing can considerably reduce the workload execution cost. Views space constraint Basic indexing space Sel=0.1% Sel=2% Sel=10% 100 Mb 190 Mb 43.99% 1.43% 0.01% 300 Mb 198 Mb 41.06% 1.52% 0.01% 500 Mb 226 Mb 41.01% 1.40% 0.01% Table 1 The Table 1 shows how the space for full indexing reduces to the low selectivity since the average utility of indexes decreases. It is remarkable that the indexes created are always used by the DBMS and each index actually reduces the overall execution cost. Besides, it is worth mentioning that there is a correlation between the workload selectivity and the best trade-off between the space used for views and that used for indexing. Figure 2 shows the workload costs, for the different ratio between space constraint on views and on index (S/VS). Figure 2 It is easily seen that high selectivity encourages indexing, while at low selectivity view materialization is more convenient.

10 10 6 Related work 6 RELATED WORK, 7 SUMMARY, LITERATURE Just a few works in the literature focus on the selection of indexes for DWs. In [6] the authors propose both an optimal algorithm and a set of thumb rules that should be adopted when the problem size is intractable. Rules, that are justified by the adoption of appropriate cost functions, state that indexes should be created on keys and on attributes involved in joins, as well as when their size fits into main memory. In [3] the problem of simultaneously choosing views and B+-tree indexes is investigated; the linear cost function adopted is very simple, and no specific optimizer model is considered. 7 Summary For the implementation of a complete DW, a set of tools must be integrated to form a concrete warehousing solution. The attention in this paper was concentrated on the most important level of the relational database modeling cycle: DW design; with the main focus to it s physical phase. Heuristic approach to the index selection problem in a data warehouse with materialized views was proposed, which experimental performance evaluation on the TPC-H benchmark data demonstrated a sound improvement results. However, it has to be expanded to the other types of indexes and join algorithms. Literature [1] R. Kimball. The data warehouse toolkit. John Wiley & Sons, [2] Informix. Administrator s Guide Informix Red Brick Decision Server, Version 6.0, November [3] M. Golfarelli, S. Rizzi, E.Saltarelli. Index selection techniques in data warehouse systems D2.R5 4 febbraio 2002 [4] G. Graefe. Query Evaluation Techniques for Large Databases. ACM Computing Surveys, 25(2):73 170, June [5] A. Gupta, V. Harinarayan, and D. Quass. Aggregate-Query Processing in Data Warehousing Environments.In Proc. 21st VLDB, Zurich, Swizerland, 1995.

11 11 A. INDEX SELECTION ALGORITHM

12 12 B. COSTS FUNCTIONS