Query Optimization in Microsoft SQL Server PDW

Transcription

1 Qery Optimization in Microsoft SQL Server PDW Srinath Shankar, Rimma Nehme, Josep Agilar-Saborit, Andrew Chng, Mostafa Elhemali, Alan Halverson, Eric Robinson, Mahadevan Sankara Sbramanian, David DeWitt, César Galindo-Legaria Microsoft Corporation ABSTRACT In recent years, Massively Parallel Processors have increasingly been sed to manage and qery vast amonts of data. Dramatic performance improvements are achieved throgh distribted exection of qeries across many nodes. Qery optimization for sch system is a challenging and important problem. In this paper we describe the Qery Optimizer inside the SQL Server Parallel Data Warehose prodct (PDW QO. We leverage existing QO technology in Microsoft SQL Server to implement a cost-based optimizer for distribted qery exection. By properly abstracting metadata we can readily rese existing logic for qery simplification, space exploration and cardinality estimation. Unlike earlier approaches that simply parallelize the best serial plan, or optimizer considers a rich space of exection alternatives, and picks one based on a cost-model for the distribted exection environment. The reslt is a high-qality, effective qery optimizer for distribted qery processing in an MPP. Client Connections User Qeries PDW Engine JDBC OLEDB ODBC ADO.NET Control Node Parse SQL Validate and Athorize Bild MPP Plan Execte Plan Retrn Data to Client SQL Server DMS Service Categories and Sbject Descriptors H.2.4 [Database Management]: Parallel databases; H.2.4 [Database Management]: Qery processing Keywords Qery Optimization 1. INTRODUCTION 1.1 Overview of SQL Server PDW One of the major trends in recent years has been the wide adoption of Massively Parallel Processing (MPP systems, i.e., distribted systems consisting of mltiple independent nodes connected by a network. MPP systems are typically sed as data warehoses, that is they are sed to manage and qery vast amonts of data. Microsoft SQL Server Parallel Data Warehose [9, 10] is a shared- PDW is short for Parallel Data Warehose. Permission to make digital or hard copies of all or part of this work for personal or classroom se is granted withot fee provided that copies are not made or distribted for profit or commercial advantage and that copies bear this notice and the fll citation on the first page. To copy otherwise, to repblish, to post on servers or to redistribte to lists, reqires prior specific permission and/or a fee. SIGMOD 12, May 20 24, 2012, Scottsdale, Arizona, USA. Copyright 2012 ACM /12/05 $ Figre 1: Microsoft SQL Server PDW. nothing parallel database appliance and is one example of an MPP system. SQL Server PDW comes in a nmber of hardware configrations, with the necessary software pre-installed and ready to se. It has a control node that manages a nmber of compte nodes (see Figre 1. The control node provides the external interface to the appliance, and qery reqests flow throgh it. The control node is responsible for qery parsing, creating a distribted exection plan, issing plan steps to the compte nodes, tracking the exection steps of the plan, and assembling the individal pieces of the final reslts into the single reslt set that is retrned to the ser. Compte nodes provide the data storage and the qery processing backbone of the appliance. The control and compte nodes each have a single instance of SQL Server RDBMS rnning on them. User data is stored in tables that are hash-partitioned or replicated tables across the SQL Server instances on the compte nodes. To execte a qery, the control node transforms the ser qery into a distribted exection plan (called DSQL plan that consists of a seqence of operations (called DSQL Operations. At a highlevel, every DSQL plan is composed of two types of operations: (1 SQL operations, which are SQL statements to be exected against the nderlying compte nodes DBMS instances, and (2 DMS operations 1 which are operations to transfer data between DBMS instances on different nodes. 1 DMS is short for Data Movement Service. 767

2 1.2 Qery Optimization in PDW Qeries exected in MPP environments tend to be complex involving many joins, nested sb-qeries and aggregations and are sally long-rnning and resorce-intensive. The goal of the qery optimizer is to find the best exection plan for a given qery, which is sally accomplished by examining a large space of possible exection plans and comparing these plans according to their estimated exection costs. Developing an indstrial-strength qery optimizer from scratch is a major ndertaking. Enmerating exection alternatives adeqately reqires an nderstanding of relational algebra and its properties, as well as deriving and identifying desirable exection plans. Effective plan selection reqires carefl modeling of data distribtions and cost estimation. For a commercial prodct, time to market is also a critical dimension to consider. Rather than starting from scratch, the PDW qery optimizer reses the technology developed for SQL Server, which has been tned and tested over a nmber of releases. This allowed s to craft in a short time a qery optimizer for parallel qeries that is rich in fnctionality and prodces high-qality plans. PDW invokes the SQL Server QO against a shell database to obtain a compact representation of the optimization search space called a MEMO [5, 6]. This search space is then agmented with statistical information on the distribtion of data in the appliance to find a parallel exection plan for the qery., taking into consideration the available statistics and the actal data distribtion in the appliance. The main idea of PDW QO can be smmarized as follows: 1. We store the metadata of the distribted tables in a shell database on a single SQL Server instance. The shell database provides the single system image of the data in the appliance. Importantly, it also stores aggregated statistical information on the ser data. 2. Using the shell database, we se the existing compilation stack of SQL Server to parse a given qery, and generate and export the space of exection alternatives (MEMO. 3. We traverse the space of exection alternatives to introdce data movement operations, and make a cost-based decision on the best exection plan for the distribted environment. The rest of the paper is organized as follows. We first give an overview of the Microsoft SQL Server PDW architectre in Section 2. We provide detailed discssion of PDW QO implementation in Section 3. Section 4 contains detailed qery example illstrating the flow of qery optimization. Finally, we conclde in Section MICROSOFT SQL SERVER PDW ARCHITECTURE OVERVIEW 2.1 Appliance The PDW appliance is composed of hardware and software architected to fnction together as one box. Mltiple servers are sed to implement scale-ot qery processing in a shared-nothing fashion. Users can only access the box throgh the ser interface of the appliance. Indstry standard hardware is sed for servers, networking components and storage arrays. This approach allows cost effective and incremental growth of the appliance by adding extra servers or storage. Additionally, as the appliance grows over time the server components can be pgraded or individally replaced with newer generations of more powerfl CPUs, memory, storage, etc. There are two distinct types of nodes that implement the qery processing fnctionality (see Figre Control Node. The control node manages the distribtion of qery exection across the compte nodes, accepts client connections to the PDW appliance and manages client athentication. In addition to containing a SQL Server instance, the control node contains additional software to spport the distribted architectre of the PDW. This incldes the engine that coordinates the data warehosing fnctions that are specific to processing parallel qeries, stores appliance-wide metadata and configration data, and manages appliance and database athentication and athorization. The control node also manages the Data Movement Service (DMS (described in Section 2.3 that rns on the appliance nodes and is the commnication layer for transfering data between the nodes in the appliance. A ser can connect to the PDW control node sing a variety of client access tools sing the drivers with connection types, sch as: ODBC, OLE DB, ADO.NET. 2. s. Each compte node is the host for a single SQL Server instance. It also rns a DMS process for commnication and data transfer with the other nodes in the appliance. Each compte node stores a portion of the ser data. Tables in a PDW appliance can either be 1. replicated on each compte node in the appliance, or 2. hash-partitioned on a specified colmn(s across the compte nodes. 2.2 Shell Database A shell database is a SQL Server database that defines all metadata and statistics abot tables, bt does not contain any ser data. From the point of view of compilation, athentication, athorization and qery optimization, a shell database is ndistingishable from one that contains actal data. Shell databases are sed by SQL Server for testing and debgging of compilation isses withot having to copy large databases. For PDW, a shell database residing on the SQL Server instance at the control node is sed to store the metadata for the ser tables partitioned across the compte nodes. It provides all the information needed to compile and generate a space of exection alternatives for qeries. In addition to table metadata, we also store in this database all information regarding sers and privileges, so that compilation can check for secrity and access rights. This enables PDW to provide the same secrity model as SQL Server, at no extra cost. The shell database also contains global statistics for all the tables in the appliance. To compte global statistics, local statistics are first compted on each node via the standard SQL Server mechanisms, and are then merged together to derive global statistics Data Movement The Data Movement Service (DMS is responsible for moving data between all the nodes on the appliance. Once instance of DMS rns on each of the control and compte nodes. Certain steps of a ser qery may reqire intermediate reslt sets to be moved from one compte node to another. In addition, sometimes intermediate reslt sets from one or more compte nodes mst be moved to the control node for final aggregations and sorting prior to retrning the reslt set to the client. PDW tilizes temporary (temp tables on the compte and control nodes as necessary to move data or store intermediate reslt sets. In some cases, qeries can be written that generate no temp tables and reslts can be streamed from the compte nodes directly back to the client that issed the qery sch qeries will not involve DMS. 768

3 2.4 The DSQL Plan and its Exection Given a ser-specified SQL qery, the PDW engine is responsible for creating a parallel exection plan (known as a DSQL plan. A DSQL plan may inclde the following types of operations: SQL Operations that are exected directly on the SQL Server DBMS instances on one or more compte nodes. DMS Operations which move data among the nodes in PDW for frther processing, e.g. moving intermediate reslt sets from one compte node to another. Temp table operations that set p staging tables for frther processing. Retrn operations which psh data back to the client. Qery plans are exected serially, one step at a time. However, a single step typically involves parallel operations across mltiple compte nodes. DSQL Plan Example: Using the TPC-H schema as an example, let s assme that the Cstomer table is hash-partitioned on c_cstkey, andtheorders table is hash-partitioned on o_orderkey and we want to perform the following join between these two tables. SELECT c_cstkey, o_orderdate FROM Orders, Cstomer WHERE o_cstkey = c_cstkey AND o_totalprice > 100 The table partitioning is not compatible with the join since Orders is not partitioned on o_cstkey. Ths, a data movement operation is reqired in order to evalate the qery. The optimizer on the control node may prodce a DSQL plan consisting of the following two steps: 1. DMS Operation that repartitions data in the Orders table on o_cstkey in preparation for the join. 2. Retrn SQL Operation that selects tples for the final reslt set from each compte node and retrns them back to the client. Once the plan has been generated, the Engine service exectes the plan by walking the list of plan steps and distribting the steps oneby-one to the compte nodes. Step 1: DMS Operation: In the example above, the first step in the DSQL plan is a DMS operation that repartitions Orders data on o_cstkey. The DMS operation specifies the (1 SQL statement reqired to extract the sorce data, (2 the tple roting policy (e.g., replicate or hash-partition on a particlar colmn, and (3 the name of a (temporary destination table. The Engine service then begins broadcasting the DMS operation from the control node to the DMS instance on each node. Upon receiving the DMS message, the DMS instance on each compte node begins exection of the data movement operation by issing the SQL statement below: SELECT o_cstkey, o_orderdate FROM Orders WHERE o_totalprice > 100 against the local SQL Server instance. Each DMS instance reads the reslt tples ot of the local SQL Server instance, rotes the tples to the appropriate DMS process by hashing on o_cstkey, and also inserts the tples it receives from other DMS instances into the specified local destination table (Temp_Table in this example. Once all of the tples from the sorce SQL statement have been inserted into their respective destinations the DMS operation is complete. Inpt Qery from PDW ser 1 PDW Parser Qery Transform T-SQL generator PDW Engine process PDW exection Transformed qery against the shell database Shell Metadata 2 Qery reslts SQL Server parser Algebrizer SQL Server Qery Optimizer DSQL Plan SQL Server process Search space (MEMO 4 PDW memo parser Qery optimizer DSQL generator Encoded search space as XML 3 XML generator Figre 2: Overview of qery optimization in PDW. Step 2: SQL Operation: After the DMS operation has completed, the Engine service moves on to the second step in the plan, which is the SQL operation that is sed to pll the reslt tples from each compte node. To perform this operation, the Engine service obtains a connection to the SQL Server instance on each compte node and isses a specified SQL statement. In this case, the SQL statement that will be exected is: SELECT c.c_cstkey, tmp.o_orderdate FROM Cstomer c, Temp_Table tmp WHERE c.c_cstkey = tmp.o_cstkey To complete the reqest, the Engine service reads the reslt tples from each compte node, packages them into the final reslt, and sends them back to the client. The qery exection is now complete. From the client s perspective, it appears as if all of the data was stored and all of the comptation took place in a single SQL Server instance on the control node. 2.5 Cost-Based Qery Optimization in PDW Figre 2 provides the high-level data flow for PDW qery optimization. The key observation that forms the basis of PDW QO is that the problem of a algebrizing inpt qeries into operator trees, and b the logical (as opposed to physical, or partition-dependent exploration done on operator trees to find plan alternatives is the same for PDW as it is for a single SQL server instance. We describe each of the QO components and their fnctionality below: 1. PDW Parser: This component is responsible for parsing the inpt qery string and creating an abstract syntax tree (AST strctre that can be validated against PDW syntax rles. Some PDW qeries may also need a few basic transformations before they are ready to be sent to SQL Server against the shell database. 2. SQL Server Compilation: After validation by the PDW parser, the qery is passed to SQL Server for compilation against the shell database. The SQL Server optimizer performs the following fnctions: (a Simplification of the inpt operator tree into a normalized form. This is inserted as the initial plan into the MEMO [5, 6] data strctre, which will hold the space of alternative plans for the qery. 769

4 (a (b Select * From Cstomer C, Orders O Where C.c_cstkey = O.o_cstkey and O.o_totalprice > 1000 Get C Join Select Get O (c Agmented (Parallel Memo Grop 6: 1. Move(3 2. Shffle( Replicate(3.2 Grop 5: 1. Move(1 2. Shffle( Replicate(1.4 Grop 4: Grop 3: Grop 2: Grop 1: Initial Memo 1. Join(1,3 1. Select(2 1. Get O 1. Get C 6. HJ(1.4, MJ(1.5, Filter( Sort( Table Scan 4. Table Scan 4. Sort Sorted Index Scan Final (Serial Memo (d 4.6. HashJoin 1.4. Table Scan (C 6.2. Shffle 3.2 Filter 2.3. Table Scan(O (e Shffle (o_cstkey: Data Select * From Orders O Movement Where O.o_totalprice > 1000 Operation TempTable: T1 Retrn: SQL Select * Operation From Cstomer C, T1 Where C.c_cstkey = T1.o_cstkey Figre 3: Parallel qery optimization flow: (a inpt qery, (b logical qery tree, (c agmented MEMO, (d best qery plan, (e final DSQL plan. (b Logical transformations on the plans in the MEMO data strctre to agment the set of choices. These are based on relational algebra rles. For instance, all eqivalent join orders are generated in this stage. (c Estimation of the size of intermediate reslts for each of the exection alternatives. These estimations are based on the size of base tables and statistics on the colmn vales. (d The implementation phase which adds physical operator (algorithms choices into the search space. The optimizer costs them and prnes the plans that do not meet established lower bonds. (e Extraction of the optimal exection plan. While the otpt of the SQL server optimizer is an optimal exection plan, the best serial plan (i.e., single-node exection plan will not prodce the best distribted exection plan. We elaborate on this later in this section. PDW does not take the best plan obtained by the SQL Server optimizer; rather the PDW QO will consme the entire space of alternatives generated by the SQL QO. 3. XML Generator: This component takes the search space generated by SQL Server optimizer represented in the MEMO data strctre as its inpt and encodes the information as XML. 4. PDW Qery Optimizer: The PDW qery optimizer is the consmer of the search space otpt from the XML generator. There is a memo parser on the PDW side which is responsible for constrcting the memo data strctre for the PDW qery optimizer. Once the memo data strctre for the PDW side is constrcted, the PDW optimizer performs bottom-p optimization (see Section 3.2 with the help of the PDW cost model. The responsibilities of the PDW qery optimizer inclde: (a Enmeration of distribted exection plans by systematically adding appropriate data movement strategies into the search space. (b Costing the alternative plans generated sing the PDW cost model. (c Choosing the optimal (minimal cost distribted exection plan. Once optimization process is completed, the DSQL generator component constrcts the qery plan for distribted exection of the qery. Example: Consider the following SQL qery: SELECT * FROM CUSTOMER C, ORDERS O WHERE C.C_CUSTKEY = O.O_CUSTKEY AND O.O_TOTALPRICE > 1000 Figre 3 visally depicts the flow of parallel qery optimization for the above qery. We first parse the inpt qery (Figre 3(a and transform it into a tree of logical operators (Figre 3(b. Traditional qery optimization is performed prodcing a MEMO. TheMEMO consists of two mtally recrsive data strctres, called grops and gropexpressions. A grop represents all eqivalent operator trees prodcing the same otpt. To redce memory reqirements, a grop does not explicitly enmerate all its operator trees. Instead, it implicitly represents all the operator trees by sing gropexpressions. A gropexpression is an operator having other grops (rather than other operators as children. As an example, consider Figre 3(c, which shows a MEMO for the qery example above (logical operators are shaded and physical operators have white backgrond. In the figre, grop 1 represents all eqivalent expressions that retrn the contents of table Cstomer (or C, for short. Some operators in grop 1 are logical (e.g., Get C, and some are physical (e.g., Table Scan, which reads the contents of C from the primary index or heap, and Sorted Index Scan, which does it from an existing secondary index. In trn, grop 4 contains all the eqivalent expressions for C O. Note that gropexpression 4.1 (i.e., Join(1,3, represents all operator trees whose root is Join, first child belongs to grop 1, and second child belongs to grop 3. In this way, a MEMO compactly represents a potentially very large nmber of operator trees. Children of physical gropexpressions point to the most efficient gropexpression in the corresponding grops. For instance, gropexpression 4.6 represents a hash join operator whose lefthand-child is the forth gropexpression in grop 1 and whose right-hand-child is the third gropexpression in grop 2. In addition to enabling memoization (a variant of dynamic programming, a MEMO provides dplicate detection of operator trees, cost management, and other spporting infrastrctre needed dring qery optimization. Additional details on the organization of the MEMO strctre can be fond in the literatre [5, 6]. Once the memo from SQL Server (i.e. the serial MEMO has been generated, it is agmented to additionally consider parallelism (see Figre 3(c. The PDW optimizer then introdces new data movement grops and operations into the MEMO based on the nderlying data distribtions. For example, see grops 5 and 6 in the Figre 3(c. Grop 5, represents the data movement of the otpt of grop 1 (i.e., the tples from C. Assming, C and O are distribtion-incompatible, this operation wold be considered by 770

5 the parallel optimizer as one of the options in order to make both C and O partition-compatible to perform the join C O. Grop 6, on the other hand, represents the data movement of the other inpt to the join, namely O. Like relational operations, logical data movement operations may have a nmber of physical implementations, sch as Shffle (re-partition of data on a colmn(s, Replication, and so on. The final exection plan, which consists of a tree of physical operators, is extracted from the MEMO (shown in Figre 3(d. THis plan is then transformed into an exectable DSQL plan that will be rn in the appliance (shown in Figre 3(e. Why Parallelizing The Best Serial Plan Is Not Enogh The optimization objectives for PDW and SQL Server are different [8]. The SQL Server optimizer is naware of the partitioning of data, while PDW has the additional task of inserting and costing data movement operations to obtain a correct and efficient parallel plan. However, the problem of arriving at a logical search space is common to both, for the most part. Hence we export the logical search space from the SQL server process into PDW for optimization based on PDW objectives. This is best illstrated with an example. Consider the TPCH database, with the Cstomer, Orders and Lineitem tables partitioned on cstkey, orderkey, andorderkey, respectively. Consider a qery that performs a join between these three tables on cstkey and orderkey. The best serial plan may prodce the following join order: Cstomer, Orders, Lineitem, in increasing order of table size. However, a better parallel plan may be obtained by choosing the following join order: Orders, Lineitem, Cstomer, i.e.joinorders and Lineitem first and then shffle (i.e., redistribte the reslt on cstkey. This plan may be better de to the co-location of Orders and Lineitem. 3. IMPLEMENTATION OF PDW QO Althogh the conceptal idea behind PDW QO is relatively simple, there were a nmber of technical challenges to address. We discss them in detail in this section. 3.1 Changes to SQL Server A few changes are needed in SQL Server to rese its logic for PDW optimization. The first change is to spport exporting the optimizer search space. We defined a new compilation entry point to reqest the optimizer MEMO, in a way similar to the showplan XML fnctionality already available in SQL Server. This new entry point also triggers the se of any PDW-specific logic in the SQL Server compilation stack. When the PDW-compilation is reqested, the otpt from SQL Server is an XML representation of the MEMO data strctre. A second change is to extend the qery srface to spport all constrcts of PDW. The goal for PDW is to be flly compatible with SQL Server. So this extension is limited to a handfl of qery hints for specific distribted exection strategies. The third change is to expand the optimizer search space to inclde some alternatives that are relevant for distribted qery exection, especially arond collocation of joins and nions. The transformation-based architectre of the qery optimizer in SQL Server allows implementing these extensions withot major changes to the framework. For very large search spaces, the SQL Server optimizer ses a timeot mechanism and does not generate all possible plans. In those cases the initial exection alternatives placed in the MEMO have a big inflence on the space considered. For PDW optimization, we seed the MEMO with exection plans that consider distribtion information of tables, for collocated operations. 3.2 Plan Enmeration The search space of possible plans is hge and by introdcing data movement (DMS operations we increase it even frther. Naïve enmeration is not likely to be sccessfl for any bt the simplest of qeries. In this section, we describe the bottom-p enmeration strategy in the PDW optimizer. While or crrent implementation employs a bottom-p search strategy, a top-down enmeration techniqe is eqally applicable to the PDW QO design. Steps 5-7 in Figre 4 depict the psedo-code for the bottom-p search strategy. A bottom-p optimizer starts by optimizing the smallest expressions in the qery, and then ses this information to progressively optimize larger expressions ntil the optimal physical plan for the fll qery is fond. Bottom-p optimizers pay special attention to physical properties that affect the ability to generate the optimal plan. Interesting properties in the PDW qery optimizer represent an extension of the notion of interesting orders introdced in System R [3]. For example, a sbplan that retrns reslts distribted on a certain colmn may be preferable to a cheaper alternative, becase later in the plan this distribtion can be leveraged to obtain a globally optimm soltion. Specifically, the PDW qery optimizer considers the following colmns to be interesting with respect to data movements: (a colmns referenced in eqality join predicates, and (b grop-by colmns. Join colmns are interesting becase they make local and directed joins possible, and grop-by colmns are interesting becase aggregations can be done locally at each node and the reslts nioned together, withot having to do local / global processing. 3.3 Cost Model To evalate the performance of a specific plan, we se a cost model that costs data movement operations. Despite the robstness that costing relational operators wold give to the cost model, costing DMS operations is a good start for developing a high qality cost based optimizer: Costing data movement operations is a sbset of the overall complete cost model for plans involving both data movement and SQL relational operations. Ths, the scope is smaller and more manageable. The development, testing and debgging of the cost model is less complex compared to the mixed cost model involving relational operations. Data movement processing times tend to dominate qeries overall exection times in PDW de to materializing data to temp tables. Ths, optimizing for data movements is expected to prodce good qality plans for a broad set of qeries. There is no eqivalent to data movement operations inside SQL Server, ths we cannot rely on SQL Server optimizer to generate the costs for these operations Cost Model Assmptions Bilding a cost model from scratch is a challenging task. Or crrent version of the cost model strives for simplicity while trying to ensre high qality. The scope of the crrent version of the cost model is to only cost DMS operations in terms of response time. With this prpose in mind the following assmptions hold: Absence of independent parallelism. The cost model assmes a seqential exection of the DSQL steps. Absence of pipelined parallelism between DSQL steps. Two steps in a consmer-prodcer relationship are not exected concrrently; intermediate reslts are always materialized. 771

6 PDWOptimizer ( 01 Parse the MEMO XML from SQL server into the PDW MEMO object. 02 Apply MEMO pre-processor rles (bottom-p. Example: Fix cardinality estimates of partial aggregates based on PDW topology. 03 Merge eqivalent grop expressions from the perspective of PDW within the grops (bottom p. 04 Derive interesting properties of grops (top-down. /*** BOTTOM-UP ENUMERATION ***/ 05 For each grop in the MEMO, in bottom-p order, do the following: 06 Enmeration step: 06.i Enmerate PDW optimization options by considering all possible inpts from child grops. If it s a base grop, add a Get operation. Apply heristic prning. 06.ii Cost based prning: As PDW grop expressions are added, keep only the overall best option and the best option per each interesting property. Do prning every time a new PDW grop expression is added into the MEMO. The maximm cardinality of the set of PDW grop expressions in a grop is (# of interesting properties + 1. //one is added to accont for overall best. 07 Enforcer step: Add PDW move grop expressions based on interesting properties in the crrent grop. Apply cost-based prning similar to step 06.ii above. /*** END BOTTOM-UP ENUMERATION ***/ 08 Extract the best overall plan by starting at the best plan in the root grop and going down the memo to obtain the optimal plan tree. 09 Apply post-optimization rles on the optimal plan tree. 10 Perform DSQL-generation by traversing the optimal plan tree bottom-p. 11 Apply post-dsql-generation rles on the DSQL plan. 12 Retrn the DSQL plan to the engine for exection. Figre 4: Psedo-code for PDW Optimizer (component 4 in Figre 2. Isolation. When costing a qery, we assme it rns in isolation. Homogeneity. Homogeneos hardware across all nodes in the appliance (one topology. Uniformity. Uniform distribtion of data across nodes. When costing an operator which is cloned and exected in mltiple independent nodes, we cold compte the cost of an operator as max 1 i N (C[i] where N is the nmber of nodes, and each component of C its exection cost on a particlar node. With the niform and homogeneity assmptions, the components of the cost vector C are assmed to be identical, ths simplifying the cost model in the sense that only one node needs to be considered Types of DMS Operations Before we present the details of or cost model, we first discss the varios physical data movement operations in PDW. There are 7 data movement operations: 1. Shffle Move (many-to-many. Rows are moved from each compte node to target table based on a hash of the vale in the specified distribtion colmn. 2. Partition Move (many-to-one. Rows are moved from each compte node to the target table on the target node (typically the control node bt this is not a reqirement. 3. Control-Node Move (From the control node to the compte nodes. A table in the control node is replicated to all compte nodes. 4. Broadcast Move. Rows are moved from each compte node to the target table on all compte nodes. 5. Trim Move. Trim move is initiated against a replicated table on all compte nodes where the destination is to a distribted table on its own nodes. Hashing will take place so that only rows that this node is responsible for will be kept. 6. Replicated broadcast. A table which is only in one compte node it is replicated via a broadcast move. Sorce rk o tw e N Reader m a s t re t p In r y e q L Q (S DMS operator Target Figre 5: A DMS operator. r k o t w e N Writer m a e tr s t tp O y p o c lk B L ( S Q SQL Server Tempdb (Temp_Table 7. Remote copy to single node. Can be either a remote copy of a replicated table (from control node or from compte node or, a Remote copy of a distribted table. Each of the DMS operations defined above is implemented by a common rntime operator, the DMS operator. The cost of the DMS operator will be different depending on the operation being implemented Cost of a DMS operator Figre 5, shows the basic strctre of a DMS operator. We can envision a DMS operator as two components with separate fnctionalities that we name the sorce, and the target. The sorce of a DMS operator is the sending side and can be broken into the following two cost sb-components: C reader : Read tples from the qery exected against SQL Server and packing them into a bffer. C network : Send the data bffers over the network. 772

7 Shffle Shffle Shffle OnOperation: CREATE TABLE temp Shffle: SELECT Retrn: SELECT (a Physical Operator Tree (b QRel RelOp Tree (c DSQL Plan (d DSQL Formatted Plan Figre 6: DSQL Generation. The data is sent asynchronosly over the network; therefore we can define the cost of the sorce component as the most expensive of its components: C sorce =max(c reader,c network. The target in a DMS operator is the receiving side and, is also broken into two cost sb-components: C writer: Unpack the tples from the bffers sent by the sending process, and prepare bffers for insertion into a temporary table. C SQLBlkCpy : Blk copy operation for insertion of the data bffers into a SQL Server temporary table. The blk insert into the temporary table is done asynchronosly, so similarly to the sorce component, the cost of the target can be defined as: C target = max(c writer, C SQLBlkCpy. The sorce and the target of a DMS operator rn in parallel on each node. Therefore, we define the cost of a DMS operation as follows: C DMS =max(c sorce, C target. Costing of an Individal Component Theoretically, the more complex the model the more accrate the estimates shold be. However, it has also been proven that, the more sophisticated the cost model is, the more sensitive it is to slight changes in data and statistics [7]. Frthermore, it becomes more difficlt to debg and maintain the cost model over time. Or crrent version of the cost model costs each individal component based on the nmber of raw bytes processed: C X = B * λ,whereb is the nmber of raw bytes and λ is the cost per byte for the specific component (C X. The constant λ is calclated via targeted performance tests after a meticlos instrmentation of the sorce code. We call the process of defining the vale of λ for each cost component cost calibration. The reslts of or cost calibration showed that there are differences in the vale of λ depending on the nmber of rows, nmber of colmns or colmn type. However, the differences observed were not significant enogh to jstify stepping p the complexity of the cost model. Therefore, the vale of λ is considered to be constant regardless of these parameters. Each cost component has its own constant vale of λ. C reader, however, was one exception and reqired two constants, denoted as λ hash and λ direct,to accont for the extra overhead that hashing has for some data move operations e.g. Shffle or Trim. The nmber of bytes B to be processed by each individal cost component depends on the distribtion properties of the inpt and otpt streams. Let Y denote global cardinality, and w the width of the row (both vales are provided by the statistics exported in the MEMO. Let N denote the nmber of nodes in the appliance. Then, nder the niformity assmption, we can compte B as: ( Y w for distribted data streams, and N (Y * w for replicated data streams. 3.4 DSQL Generation Finally, once the qery exection plan is selected by the PDW qery optimizer, it mst be translated to DSQL format to be able to rn it on the actal compte nodes. Unlike other MPP systems, e.g., GreenPlm [2], instead of sending an operator tree to each compte node, SQL Server PDW sends a SQL statement to the compte nodes. The SQL statements are exected against the nderlying compte nodes standard DBMS instances, and data movement operations are sed to transfer data between DBMS instances on different nodes. This is a similar approach to AsterData [1]. Performing DSQL generation reqires taking an operator tree and translating it back to SQL. We employ the QRel programming framework [4], which encapslates the knowledge of mapping relational trees to qery statements. The process is depicted in Figre 6. A physical operator tree prodced by the PDW qery optimizer is first converted into a RelOp tree [4], which has strctre similar to SQL server algebrizer otpt tree. RelOp tree is then converted into a PIMOD AST by the QRel library. Finally, the PIMOD script generator generates T-SQL string from the reslting AST. For more details on this process, we refer the reader to [4]. 4. QUERY OPTIMIZATION EXAMPLE Figre 7 is an example of the parallel plan generated by PDW for Q20 in TPC-H. The figre also shows the DSQL plan generation otpt for each section of the parallel qery plan, separated by a red line in the Figre. For readability we don t show the temporary tables in the qery plan, however, each data move operation is materializing the otpt of each DSQL step; in other words, DSQL steps are not pipelined. Q20 is interesting from the point of view that exercises key featres in qery optimization like: sb-qery removal, sb-qery into join transformation, join transitivity closre detection etc. DSQL steps 0 and 1. The semi-join between lineitem and part transformed into a join and a grop by on p_parkey is the reslt of de-correlating the two most inner sb-qeries (SQ2 and SQ3, transforming them into joins, and detecting the transitivity closre between ps_partspp, part and lineitem; this strategy allows the early filtering of lineitem, by joining it with part. De to the high selectivity predicate on part, a broadcast join is the preferred option in this case (DSQL step 0; a shffle of lineitem is also considered bt reslts into a more expensive strategy. After the broadcast of part, we shffle the otpt of the join between part and lineitem (DSQLstep 1 on l_partkey to compte the grop by aggregation (sm(l_qantity in a distribted manner. Note 773

8 that the cost model opts for a local-global transformation of the grop by aggregation becase is cheaper from a data move point of view de to the redction of the data set performed by the local grop by. DSQL step 2. The global aggregation sm(l_qantity takes place and we perform the semi-join between partspp and part. Finally, we perform a shffle of the reslt set on ps_sppkey to perform a distribted grop by to execte the final semijoin between spplier and partspp; that is the reslt of transforming SQ1 into a join. Again, a local grop by strategy is chosen to redce the amont of data being commnicated. DSQL step 3. Is the last step of the parallel plan in which reslts are commnicated to the ser. 5. CONCLUSION Throgh the se of technology developed for SQL Server, the PDW QO goes beyond simple predicate pshing and join reordering, and incorporates a nmber of advanced qery optimization techniqes. These techniqes inclde things sch as contradiction detection, redndant join elimination, sbqery nnesting, and oterjoin reordering. The cost model of PDW QO is specially crafted to reflect the distribted environment, and its se on a rich space of alternatives prodces mch higher-qality plans than simply parallelizing the best serial plan. Another important aspect of technology rese was that it shortened the time to bild a cost-based optimizer for PDW. Sccessfl delivery reqired designing the right abstractions and interfaces. The qality and effectiveness of the reslt validate the approach, which qickly incorporated state of the art commercial qery optimization in the PDW prodct. 6. REFERENCES [1] Aster Data. [2] Greenplm. [3] M. M. Astrahan, M. W. Blasgen, D. D. Chamberlin, K. P. Eswaran, J. Gray, P. P. Griffiths, W. F. K. III, R. A. Lorie, P.R.McJones,J.W.Mehl,G.R.Ptzol,I.L.Traiger,B.W. Wade, and V. Watson. System R: Relational approach to database management. ACM Trans. Database Syst., 1(2:97 137, [4] M. Elhemali and L. Giakomakis. Unit-testing qery transformation rles. In DBTest, pages 1 6, [5] G. Graefe. The Cascades framework for qery optimization. Data Engineering Blletin, 18(3, [6] G. Graefe and W. J. McKenna. The Volcano optimizer generator: Extensibility and efficient search. In ICDE, [7] J. R. Haritsa. The picasso database qery optimizer visalizer. PVLDB, 3(2: , [8] W. Hong and M. Stonebraker. Optimization of parallel qery exection plans in XPRS. Technical Report UCB/ERL M91/50, EECS Department, University of California, Berkeley, [9] Microsoft Corporation. Microsoft SQL Server PDW. data-warehosing/pdw.aspx. [10] R. V. Nehme and N. Brno. Atomated partitioning design in parallel database systems. In SIGMOD,

9 QUERY 20 Step bondary Logical operation Data move operation Select s_name, s_address from spplier, nation where s_sppkey in ( select ps_sppkey SQ1 from partspp where ps_partkey in ( select p_partkey SQ2 from part where p_name like 'forest%' PDW parallel plan and ps_availqty > ( select SQ3 0.5 * sm(l_qantity from lineitem where l_partkey = ps_partkey and l_sppkey = ps_sppkey and l_shipdate >= ' ' and l_shipdate < DATEADD(year,1,' ' and s_nationkey = n_nationkey and n_name = 'CANADA' order by s_name; GlobalGB Shffle (ps_sppkey LocalGB JOIN (ps_partkey = l_partkey JOIN (s_sppkey = l_sppkey JOIN (n_nation = s_nation (SUPPLIER (NATION SQL Generation SQL Generation DSQL step 3 SELECT T1_2.s_name AS s_name, T1_2.s_address AS s_address FROM (SELECT T2_1.n_nationkey AS n_nationkey FROM [tpch].[dbo].[nation] AS T2_1 WHERE (T2_1.n_name = CAST ('CANADA' AS VARCHAR (6 AS T1_1 INNER JOIN (SELECT T2_2.s_nationkey AS s_nationkey, T2_2.s_name AS s_name, T2_2.s_address AS s_address FROM (SELECT T3_1.ps_sppkey AS ps_sppkey FROM [tempdb].[dbo].[temp_id_3] AS T3_1 GROUP BY T3_1.ps_sppkey AS T2_1 INNER JOIN [tpch].[dbo].[spplier_repl] AS T2_2 ON (T2_2.s_sppkey = T2_1.ps_sppkey AS T1_2 ON (T1_2.s_nationkey = T1_1.n_nationkey ORDER BY T1_2.s_name ASC SELECT T1_1.ps_sppkey AS ps_sppkey DSQL step 2 FROM (SELECT T2_1.ps_sppkey AS ps_sppkey FROM [tpch].[dbo].[partspp] AS T2_1 INNER JOIN (SELECT SUM(T3_1.col1 AS col, MAX(T3_1.col AS col1, T3_1.l_sppkey AS l_sppkey FROM [tempdb].[dbo].[temp_id_2] AS T3_1 GROUP BY T3_1.p_partkey, T3_1.l_sppkey AS T2_2 ON ((T2_2.col1 = T2_1.ps_partkey AND (T2_2.l_sppkey = T2_1.ps_sppkey AND (T2_1.ps_availqty > (CAST ((0.5 AS DECIMAL (1, 1 * T2_2.col GROUP BY T2_1.ps_sppkey AS T1_1 GlobalGB (PARTSUPP BROADCAST Shffle (l_partkey LocalGB JOIN (p_partkey = l_partkey (LINEITEM SQL Generation DSQL step 1 SELECT T1_1.p_partkey AS p_partkey, T1_1.col1 AS col, T1_1.l_sppkey AS l_sppkey, T1_1.col AS col1 FROM (SELECT SUM(T2_2.l_qantity AS col, MAX(T2_2.l_partkey AS col1, T2_1.p_partkey AS p_partkey, T2_2.l_sppkey AS l_sppkey FROM [tempdb].[dbo].[temp_id_1] AS T2_1 INNER JOIN (SELECT T3_1.l_partkey AS l_partkey, T3_1.l_qantity AS l_qantity, T3_1.l_sppkey AS l_sppkey FROM [tpch].[dbo].[lineitem] AS T3_1 WHERE ((T3_1.l_shipdate >= CAST (' ' AS DATE AND (T3_1.l_shipdate < CAST (' :00:00.000' AS DATETIME AS T2_2 ON (T2_1.p_partkey = T2_2.l_partkey GROUP BY T2_1.p_partkey, T2_2.l_sppkey AS T1_1 GROUP BY SQL Generation DSQL step 0 SELECT T1_1.p_partkey AS p_partkey FROM (SELECT T2_1.p_partkey AS p_partkey FROM [tpch].[dbo].[part] AS T2_1 WHERE (T2_1.p_name LIKE CAST ('forest%' AS VARCHAR (7 GROUP BY T2_1.p_partkey AS T1_1 (PART Figre 7: Parallel qery plan for TPCH qery