MQOP - A Tiny Reference to the Multiple- Query Optimization Problem

Transcription

1 MQOP - A Tiny Reference to the Multiple- Query Optimization Problem MQOP - Una pequeña referencia al problema de optimización de Múltiple-Query Juan Felipe García* Abstract The multiple query optimization problem (MQOP), consists of select the execution plans such that its cost be the minimum and answer all the queries involved, typically it do so, finding the common tasks to be executed, just one time, and so, getting an improvement in the global execution time. This problem takes power of its applications in business, artificial intelligence, engineering design and many other areas where deductive and relational databases are used as well. This paper shows a a starting reference by commenting some relevant papers in this particular topic. Key words: Multiple-query optimization (MQO), database query processing, heuristics techniques, subexpression identification, database management, algorithm complexity. Received: 03/03/08. Evaluated: 28/03/08. Accepted: 14/04/08. * School of Computer Engineering - National University of Colombia 49

2 I. INTRODUCTION The multiple-query optimization problem (MQOP) is a well-known database research problem [1], typically, a query have more than one execution plan, so, the database optimization it is done by getting the cheaper cost, when the problem is extended to multiple queries, it's necessary to seek this minimum cost while all plans are considered together and the common tasks are marked to be executed just one time. The problem was presented in the 1980's, Sellis was one of the first in formulated it in [14], where the problem is divided into two orthogonal phases: The find of the common tasks among a set of queries will be the first phase, in [9] is shown that up to ten queries, to get a near-optimal global plan can be generated very efficiently. The second phase, typically the most time expensive, is where one, and only one plan from every query is chosen, and so, the global plan is generated. This phase can be studied independently from de first one, and it is NPHard as it is shown in [13]. A very good example of the problem has been shown in [1], where multi-plans are showed and self-explain what a query optimizer do, in the formulation section we will show another example referred to the problem. This is an active studied area, both in relational and deductive database, but in the beginning, this topic was opened by the research in deductive databases [2]. An important survey to the applications of the MQOP can be found in [11]. II. BASIC DEFINITIONS It is important to have a context before to attack the MQOP, we will see next, some relevant definitions to clarify the concepts when the problem it's defined and resolved. Query: Is a sentence utilized to do information retrieval from a database using boolean and specialized operators. Execution plan: A way to get a correct and exact answer to a determinate query. Global plan: Set of plans that answer every and each one of a set of queries. Data stream: Is used to deliver up-to-date information [16]. An interesting area where the MQOP takes special relevance is when the continuos queries are required, for example weather forecasts. III. PROBLEM FORMULATION When a query it's been optimized, the first job to do its to find every possible solution plan containing a set of tasks, which when executed in a certain order produce the answer of the query [1]. Each task have a cost, therefore, each plan also have a cost, associated to the sum of the costs of its tasks. Obviously, that's measure the fitness of the execution plan and the one with the minimum cost will be chosen as the optimum one. When the problem it's extended to multiple queries the problem grows to an NP-Hard problem since it should find the set of plans that produce the answer of each query, also called the global plan, such that its cost will be minimum [13]. The following example illustrates the problem. Let's have the four queries represented in the Figure 1, each one have its own execution plans, discriminated by its particular tasks, there are nine plans but just six different tasks, because some task are common to some plans. TABLE I. Example cost per task Relational database: The term relational database was originally defined and coined by Codd and it refers to a relational view of data, where the term relation is used in a mathematical sense, conforming a relational model, and refers to a database's data and schema [5]. Deductive database: In the survey from Ramakrishnan and Ullman [10], it is shown the following definition: Is a management system whose query language and storage structure are designed around a logical model of data......it is easy to see deductive database system as advanced form of relational system. The naive idea it's to calculate all the possibles combinations to answer the queries, so, having the cost of every single task as we see in the TABLE I, we can start to get possible answers, for example, if we choose the plans 50

3 A1, B1, D1 and C1, we will have a cost of four for the A1 plan, for the plan C1 we have a cost of nine, but be aware that the tasks T2 and T3 are common to those plans, so the database should execute those tasks only one time, in fact, for those plans the database will execute whole the six tasks possible but it should do it just one time. For this example the possibilities of choose different plans are 24, for each one is necessary to calculate the cost having care of the common tasks; in this case the minimum cost is getting by choose the plans A1, D2, B2 and C2 with a cost of nine (the combination A3, B2, C2 and D2 has a cost of nine as well). In the TABLE II are all the possible execution times that the multiple-query could take, in a first look, we tend to think that to compute all those times in twelve it's easy, but actually it is a NP-Hard problem, because it should be compared every task plan with every other task in the others plans, and then it is possible to add the times to get the total time, all of this just to compute one row for the table. Now, be aware that a set of execution plans for an average query could have dozens of plans, so, this is an interesting combinatorial problem. A. PROBLEM COMPLEXITY Consider the following decision problem presented in [13]: Given n sets of access plans 1; 2;...; nwith i= {P i1; P i2;... ; P iki} being the set of possible plans for processing Q i; 1 i n; and a constant K. Is there a global access plan GP such that the cost of Gp is less than or equal to K? There is clear that the problem belongs to the NP as the formulation example showed it, but it is clear also that the original problem is NP-Hard if the above decision problem is NP-Complete [7], in [13] there is a complete proof of this, and we find a good justify for the main research in the multiple-query optimization since it is touching two so active areas in the actual research community. IV. SOME APPROACHES We will present some solutions, starting with an A* algorithm near to [6] and [15], since in [1] are presented as the best know MQO algorithms in the literature, then we will show some other techniques, also heuristics but with some advantages; we will show an approach with genetic algorithms, some with graph theory and one with clustering ideas. A. A-STAR ALGORITHM FOR MQO This is a heuristic technique based on states, well defined in [6] and [15]. It has an initial state where no plan has been yet selected for any queries, intermediate states where some plans are assigned and a final state, when all queries are assigned with a plan [1]. This algorithm is well known as optimal in terms of the number of states expanded during search, but it can easily run out of memory when the size of the problem gets larger [6]. Figure 1. An example for multiple-query 51

4 In general, the algorithm set an upper bound is search values lower than that bound and when all queries have a plan associated it give a solution. B. DIVIDE AND CONQUER This early proposal consists in by changing the architecture of the well-know single-query optimization, get an answer to the MQOP, this technique was proposed by Chakravarthy in [2], who is one of the earliest researcher in this problem. His idea is to distribute the functionality of the single optimizer, organizing the strategy space of multiple queries. TABLE II. All possible values of time execution if a task is just executed in one time then do so for some iterations and returns the better one found in that iterations, like we said before, it has a very desirable execution time but the error bound can grow so fast. D. GA APPROACH The genetic algorithms (GA), are used very often to resolve NP problems, it is a metaheuristic technique that simulate the evolution concept of biology, using genes, containing part of an instance of a solution called chromosome. This technique was introduced by Holland in [8]. In [1] it is well presented the model for GA of MQOP, in general, a chromosome represents a solution as well that every gene in a chromosome represents a plan to the corresponding query. The fitness function is simply the inverse of the total execution time of all the tasks in the selected plans for the queries, the chromosome representation can be viewed as shown in [1] in Figure 2. Since all plan are valid and real, after perform genetic operation like mutation or crossover, every new solution will be valid for the MQOP, therefore this is a nice approach to solve the problem in a good time. Figure 2. Bayir's chromosome and its genes representation E. ADAPTATIVE METHOD C. GREEDY ALGORITHM This technique was often used for this kind of problems because it has a nice execution time, but it is important to be aware of its heuristic, since this algorithm just take some possible answers and search for the best one there, This method explained in [16], it's useful when we are dealing with continuous queries. The main idea is based in the possibility to consider just two cases when common operators generate common results, and those cases are: 1) If the queries are triggered by the same event. 52

5 2) If the queries are executed by similar execution patterns. In both cases this approach build something called base group, that is the improvement proposed over the others algorithms, and it is a cluster of queries or execution patterns depending of the case, for being used in the future queries without the overload of the database engine. This approach is interesting since the multiplequeries not necessary are known at the same time. F. MULTIGRAPH This technique is well studied in [3], [2] and [4]. This last one defines a multigraph as folowing: A multigraph G(R, SE, JE) is a graph with nodes R, and edges SE and JE where: 1) A node, r R, of the multigraph represents a relational or an intermediate result derived from relational operation(s). 2) A selection edge, se SE, loops on a node and i represents a selected operation on that relation. A selection edge is labeled by query ID and selection condition(s). 3) A join edge, je JE, between two relations i represents a join operation. A join edge is labeled by query ID and join condition(s). The main idea of this technique is to use the graph theory to make easily the search for common tasks between plans of a set of queries, having rules for transform the graphs depending of some possibilities for commonalities. Actually, there are four commonalities between two given conditions [4]: 1) Nothing in common. 2) Identical. 3) Subsumption. 4) Overlap. For example, a number greater than ten can be greater than hundred also, so, one condition is subsumed by the other one, in the same way a number greater than ten could be less than hundred, so, there is a overlap of the those conditions. G. AND-OR DAG AND THE VOLCANO ALGORITHMS In the way that the last approach, here, there is a solution using graphs, but this one use directed acyclic graph whose nodes can be divided into AND-nodes and ORnodes [12]. In this approach the AND-nodes represents operations and the ORnodes represents common tasks, so, after the graph was build, in this case is a tree, the OR- nodes are identified and then some kind of algorithm will choose the optimal nodes. The algorithm used in [12] is the volcano algorithm; this one is based on reusing results computed for other parts of the query and has two implementations the Sh and the RU algorithms. V. CONCLUSION The multiple-query optimization problem is a well-know and well-studied problem with a great relevance in many areas, it is important in database theory and in the combinatorial theory as well. Since the problem has been taken from so different ways to solve it, it is an interesting NP-Hard problem, and clearly the research in this topic will still active in the future. REFERENCES [1] M. A. Bayir, I. H. Toroslu, and A. Cosar, Genetic algorithm for the multiple-query optimization problem, Systems, Man, and Cybernetics, Part C: Applications and Reviews, IEEE Transactions on, vol. 37, pp , [2] S. Chakravarthy, Divide and conquer: A basis for augmenting a conventional query optimizer with multiple query-processing capabilities, in Data Engineering, Proceedings. Seventh International Conference on, 1991, pp [3] U. S. Chakravarthy and J. Minker, Multiple query processing in deductive databases using query graphs. Morgan Kaufmann Publishers Inc., 1986, pp [4] F.-C. Chen and M. Dunham, Common subexpression processing in multiple-query processing, Knowledge and Data Engineering, IEEE Transactions on, vol. 10, pp , [5] E. F. Codd, A relational model of data for large shared data banks, Commun. ACM, vol. 13, pp , [6] A. Cosar, E.-P. Lim, and J. Srivastava, Multiple query optimization with depth-first branch-and-bound and dynamic query ordering. Washington, D.C., United States: ACM, 1993, pp [7] M. R. Garey and D. S. Johnson, Computers and Intractability: A Guide to the Theory of NP- Completeness. W. H. Freeman n& Co.,

6 [8] J. Holland, Adaptation in natural and artificial systems, University of Michigan press, Ann Arbor, MI, [9] P. Kalnis and D. Papadias, Multi-query optimization for on-line analytical processing, Information Systems, vol. 28, pp , Jul [10] R. Ramakrishnan and J. D. Ullman, A survey of deductive database systems, The Journal of Logic Programming, vol. 23, pp , May [11] A. Rosenthal and U. S. Chakravarthy, Anatomy of a modular multiple query optimizer. Morgan Kaufmann Publishers Inc., 1988, pp [12] P. Roy, S. Seshadri, S. Sudarshan, and S. Bhobe, Efficient and extensible algorithms for multi query optimization, SIGMOD Rec., vol. 29, pp , [13] T. Sellis and S. Ghosh, On the multiple-query optimization problem, Knowledge and Data Engineering, IEEE Transactions on, vol. 2, pp , [14] T. K. Sellis, Multiple-query optimization, ACM Trans. Database Syst., vol. 13, pp. 2352, [Online]. Available: [15] K. Shim, T. K. Sellis, and D. S. Nau, Improvements on a heuristic algorithm for multiple-query optimization, Data Knowledge Engineering, vol. 12, no. 2, pp , citeseer.ist.psu.edu/shim94improvements.html. [16] Y. Watanabe and H. Kitagawa, Adaptive query optimization method for multiple continuous queries, in Data Engineering Workshops, st International Conference on, 2005, p Scientific and Technological Research Paper. Peer reviewed. Author Juan Felipe García School of Computer Engineering - National University of Colombia Av , Bogotá D.C., Colombia. jfgarciapen@unal.edu.co 54