Query Optimization Concepts in Distributed Database

Transcription

1 Internationa Journa of Engineering Technoogy Science and Research Voume 2 Issue 7 Juy 2015 Query Optimization Concepts in Distributed Database Rudrani sharma, Er. Pushpnee Verma, Er. Sachin Chaudhary Department Of Computer Science & Engineering Bhagwant Institute of Technoogy, Muzaffarnagar Abstract Query optimization is an important part of database management system. In this paper, through the research on query optimization technoogy, based on a number of optimization agorithms commony used in distributed query, It aims to arrive at an optima query processing pan for a given distributed query. As per the approach, the query pans having the required data residing cose to each other are considered more efficient and, therefore, these generated query pans woud resut in efficient query processing.. Keywords: Database, query processing, distributed query strategy, system mode, query processing cost, cost measures. Introduction In recent years, with the deveopment of computer network and database technoogy, distributed database is more and more widey used; with the expanding appication, data queries are increasingy compex, the efficiency requests are increasingy high, so query processing is a key issue of the distributed database system. In a distributed database environment, data stored at different sites connected through network. A distributed database management systems (DDBMS) support creation and maintenance of distributed database. The research iterature proposes a wide variety of query optimization agorithms. Yu/chang give comprehensive overviews on various query optimization techniques for distribute database management system [20]. However, these overviews do not attempt to deveop a mode of query optimization that expains and presents the agorithms in a uniform way. This understanding in case we want to change or extend existing agorithms to adapt them to new requirements. In this research we consider query processing agorithms for a Distributed Database system. There has been many research done on distributed query processing methods (see [2],[3]). Increased reiabiity and performance can aso be attained with a distributed database. A database systems must be abe to respond to requests for information from the user i.e. process queries. How a DBMS processes queries and the methods it uses to optimize their performance are topics that wi be covered in this paper. In certain sections of this paper, various concepts wi be iustrated with an exampe. Since many optimization agorithms differ in their computationa behavior whie refecting aspects of the impementation environment at the same time, it is the purpose of this paper to understand a of them by few simpe concepts. Finay, we summaries our findings and discuss future work. Genera aspects of optimization To provide a better understanding of what we mean by the term query, query processing and query optimization. Further we discuss the agorithms of query optimization that can found in a optimization agorithms described in the papers. Definitions And Exampes A. What Is A Query? A database query is the instructing a DBMS to update or retrieve specific data to/from the physicay stored medium. The actua updating and retrieva of data is performed through various ow-eve operations. Exampes of such operations for a reationa DBMS can be reationa agebra operations such as project, join, seect,cartesian product, etc. B. The Query Processor There are three phases that a query passes through during the DBMS processing of that query: 60 Rudrani sharma, Er. Pushpnee Verma, Er. Sachin Chaudhary

2 Internationa Journa of Engineering Technoogy Science and Research Voume 2 Issue 7 Juy Parsing and transation 2. Optimization 3. Evauation Most queries submitted to a DBMS are in a higheve anguage such as SQL. During the parsing and transation stage, the human readabe form of the query is transated into forms usabe by the DBMS. These can be in the forms of a reationa agebra expression, query tree and query graph. Consider the foowing SQL query: SELECT make FROM vehices WHERE make = Toyota. Query Graph After parsing and transation into a reationa agebra expression, the query is then transformed into a form, usuay a query tree or graph that can be handed by the optimization engine. The optimization engine then performs various anayses on the query data, generating a number of vaid evauation pans. From there, it determines the most appropriate evauation pan to execute. After the evauation pan has been seected, it is passed into the DMBS query-execution engine (aso referred to as the runtime database processor), where the pan is executed and the resuts are returned. B.1- Parsing and Transating the Query The first step in processing a query submitted to a DBMS is to convert the query into a form usabe by the query processing engine. High-eve query anguages such as SQL represent a query as a string, or sequence, of characters. Certain sequences of characters represent various types of tokens such as keywords, operators, operands, itera strings, etc. Like a anguages, there are rues (syntax and grammar) that govern how the tokens can be combined into understandabe (i.e. vaid) statements. The primary job of the parser is to extract the tokens from the raw string of characters and transate them into the corresponding interna data eements (i.e. reationa agebra operations and operands) and structures (i.e. query tree, query graph).the ast job of the parser is to verify the vaidity and syntax of the origina query string. B.2- Optimizing the Query In this stage, the query processor appies rues to the interna data structures of the query to transform these structures into equivaent, but more efficient representations. The rues can be based upon mathematica modes of the reationa agebra expression and tree (heuristics), upon cost estimates of different agorithms appied to operations or upon the semantics within the query and the reations it invoves. Seecting the proper rues to appy, when to appy them and how they are appied is the function of the query optimization engine. B.3- Evauating the Query The fina step in processing a query is the evauation phase. The best evauation pan candidate generated by the optimization engine is seected and then executed. (Note that there can exist mutipe methods of executing a query. Besides processing a query in a simpe sequentia manner, some of a query s individua operations can be processed in parae either as independent processes or as interdependent pipeines of processes or threads. Regardess of the method chosen, the actua resuts shoud be same.) C. Query Processing Query processing is defined as the activities invoved in parsing, vaidating, optimizing and executing a query. The main aim of query processing is Transform query written in high-eve anguage (e.g. SQL), into correct and efficient execution strategy expressed in ow-eve anguage (impementing Reationa Agebra) and to find information in one or more databases and deiver it to the user quicky and efficienty. D. Query Optimization Query optimization is defined as the activity of choosing an efficient execution strategy for processing a query. Query optimization is a part of query processing. The main aims of query optimization are to choose a 61 Rudrani sharma, Er. Pushpnee Verma, Er. Sachin Chaudhary

3 Internationa Journa of Engineering Technoogy Science and Research Voume 2 Issue 7 Juy 2015 transformation that minimizes resource usage, Reduce tota execution time of query and aso reduce response time of query. Distributed Query Processing Methodoogy: It contains four stages which are as foows: Query decomposition Normaization manipuate query quantifiers and quaification Anaysis detect and reject incorrect queries possibe for ony a subset of reationa cacuus Simpification eiminate redundant predicates Restructuring cacuus query agebraic query more than one transation is possibe use transformation rues data ocaization Input: Agebraic query on distributed reations Determine which fragments are invoved Locaization program substitute for each goba query its materiaization program optimize Goba query optimization Fragment query Find the best (not necessariy optima) goba schedue Minimize a cost function Distributed join processing 62 Rudrani sharma, Er. Pushpnee Verma, Er. Sachin Chaudhary

4 Internationa Journa of Engineering Technoogy Science and Research Voume 2 Issue 7 Juy 2015 > Bushy vs. inear trees > Which reation to ship where? > Ship-whoe vs ship-as-needed Decide on the use of semijoins > Semijoin saves on communication at the expense of more oca processing. Join methods > nested oop vs ordered joins (merge join or hash join) Data ocaization Input: Agebraic query on distributed reations Determine which fragments are invoved Locaization program substitute for each goba query its materiaization program optimize system mode Three types of modes Physica modes: capture the hardware composition of a system in terms of computers and other devices and their interconnecting network; Architecture modes: define the main components of the system, what their roes are and how they interact (software 2 system, what their roes are and how they interact (software architecture), and how they are depoyed in a underying network of computers (system architecture); Fundamenta modes: forma description of the properties that are common to architecture modes. Three fundamenta modes: interaction modes, faiure modes and security modes Cost based optimization Main Consideration for Query Optimizition Communication cost If there is severa copies of a reation, decide which copy to use Amount of data being shipped Reative processing speed at each site Site seection Goba pan Incudes severa oca pans (subqueries) If response time is critica, subqueries can be carried out in parae Loca pans constructed by optimizer of each site Cost measures n Cost is generay measured as tota eapsed time for answering query Many factors contribute to time cost disk accesses, CPU, or even network communication n Typicay disk access is the predominant cost, and is aso reativey easy to estimate. Measured by taking into account Number of seeks * average-seek-cost + Number of bocks read * average-bock-read-cost + Number of bocks written * average-bock-write-cost Cost to write a bock is greater than cost to read a bock data is read back after being written to ensure that the write was successfu Assumption: singe disk Can modify formuae for mutipe disks/raid arrays 63 Rudrani sharma, Er. Pushpnee Verma, Er. Sachin Chaudhary

5 Internationa Journa of Engineering Technoogy Science and Research Voume 2 Issue 7 Juy 2015 Or just use singe-disk formuae, but interpret them as measuring resource consumption instead of time For simpicity we just use the number of bock transfers from disk and the number of seeks as the cost measures t T time to transfer one bock t S time for one seek Cost for b bock transfers pus S seeks b * t T + S * t S We ignore CPU costs for simpicity Rea systems do take CPU cost into account We do not incude cost to writing output to disk in our cost formuae Severa agorithms can reduce disk IO by using extra buffer space Amount of rea memory avaiabe to buffer depends on other concurrent queries and OS processes, known ony during execution We often use worst case estimates, assuming ony the minimum amount of memory needed for the operation is avaiabe Required data may be buffer resident aready, avoiding disk I/O But hard to take into account for cost estimation Concusion Distributed query optimization is more compex that centraized query processing, since (i) bushy query trees are not necessariy a bad choice, (ii) one needs to decide what, where, and how to ship the reations between the sites Query optimization searches the optima query pan (tree) For N reations, there are O(N!) equivaent join trees. To cope with the compexity heuristics and/or restricted types of trees are considered There are two main strategies in query optimization: randomized and deterministic (Few) semi-joins can used to impement a join. The semi-joins requires more operations to perform, however reduces the data transfer rate INGRES, System R, and Hi Cimbing agorithms are used to optimize queries References [1] C.Yu, Z M Ozoyogu, K. Kam, Optimization of Distributed Tree Queries, J.Comput. Sys. Sci, Vo 29, No 3, pp , [2] S.Ceri, G. Poogatti, Distributed Database Principes and Systems, Mc GrawHi. [3] A. Hameurain, F. Morvan, Evoution of Query Optimization Methods, Trans. on Large Scae Data and Knowedge Cent. Syst.I, LNCS 5740, pp , [4] R. Ghaemi, AM Fard, Md. NB Suaiman, Towards Optima Query Execution in Data Grids, Advanced Technoogies, pp 57-72, [5] A. Ajanaby, E. Abuerub, M.Odeh, A Survey of Distributed Query Optimization, The Internationa Arab Journa of Information Technoogy, Vo2, No.1, [6] Craig S. Muins, Distributed Query Optimization, Technica Support [7] D.Abduah, Query Optimization in Distributed Databases, [8] S. Rho, T. March, Optimizing Distributed Joins Queries: A Genetic Approach, Annes, of Operations Research, Science Pubishers, pp , Figure 5. Diagram for Hybrid GA-ACO [30] Specia Issue: Proceedings of 2nd Internationa Conference on Emerging Trends in Engineering and Management, ICETEM 2013 [9] M.Chen, P.Yu, Using Join Operations as Reducers in Distributed Query Processing, Proceedings of 2nd Int. Symp. on Databases in Parae and Distributed System, Juy [10] PMG Apers, AR Henver, SB Yao, Optimization Agorithms for Distributed Queries, IEE Transactions on Software Engineering, Vo. 9, no.1, pp 57-68, January [11] E. Sevinc, A. Cosar, An Evoutionary Genetic Agorithm for Optimization of Distributed Database Queries,The Computer Journa, Vo. 54, No.8, Rudrani sharma, Er. Pushpnee Verma, Er. Sachin Chaudhary

6 Internationa Journa of Engineering Technoogy Science and Research Voume 2 Issue 7 Juy 2015 [12] C.Wang, M.Chen, On Compexity of Distributed Query Optimization, IEEE Transactions on Knowedge and Data Engineering, Vo. 8, No. 4, August [13] PA Bernstein, N Goodman, E.Wong, C. Reeve, Query Processing in a System for Distributed Database (SDD -1), ACM Trans. Database Sys., Vo. 6, No. 4, pp , December [14] MS Chen, PS Yu, Intereaving Join Sequence with Semijoins in Distributed Query Processing, IEEE Transac tions, Issue 5, Vo. 3, August [15] S. Pramanik, D. Vineyard, Optimizing Join Queries in Distributed Database IEEE Trans, Software Engg. Vo. 14, pp , September [16] D. Sukheja, Umesh kr. Singh, A Nove Approach of Query Optimization for Distributed Database System, IJCSI, Vo. 8, Issue 4, No. 1, Juy [17] AN Swami, A. Gupta, Optimization of Large Join Queries in Distributed Database, Proc. of ACM -SIGMOD Conference on Management of Data, pp 8-17, [18] YC Kang, Randomized Agorithms for Query Optimization, PhD Thesis, University of Wisconsin-Madison, 1991, Technica Report # [19] M. Hussein, F. Morvan, A. Hameurain, Dynamic Query Optimization, 19th Int. Conf. on Parae and Distributed Computing Systems, ISCA, [20] D. Kossman, The State of Art in Distributed Query Processing ACM Computing Survey, pp , [21] LF Lohman, GM Mackert, R* Optimizer Vaidation and Performance Evauation for Distributed Queries, Proc. of 12th Int. Conf. on VLDB, pp , [22] D. Kossman, K. Stocker, Iterative Dynamic Programming: A New Cass of Query Optimization Agorithm, ACM Transactions on Database Systems, [23] La Fortune, Wong, Query Optimization>>>> [24] M. Chen, P.Yu, Combining Join and Semi Join Operations for Distributed Query Processing, IEEE Transactions on Knowedge and Data Engineering., Vo. 5, No. 3, [25] M. Xifeng, F.Yuanyuan, Distributed Database System Query Optimization Agorithm Research, IEEE Int. Conf. on Computer Science and Information Technoogy, Vo.8, pp , [26] M. Gregory, Genetic Agorithm Optimization of Distributed Database Queries, IEEE Word Congress on Computationa Inteigence, pp , [27] M. Naohoghdem, Query Optimization in Distributed Database using Hybrid Evoutionary Agorithm, Int. Conf. on Information retrieva and Knowedge Management, pp , Rudrani sharma, Er. Pushpnee Verma, Er. Sachin Chaudhary