Example. Finally, Assuming the following access paths. Compute cost for. Compute cost for. Compute cost for

Transcription

1 376a. Database Design Dept. of Computer Science Vassar College Class 18 Cost-based query optimization algorithms News Homework #4 due today Homework # , 18.14, 18.22, 19.14, Don t forget the project. 1 2 Definitions used for costs calculations More definitions Per table r - number of tuples R - average record size (bytes) bfr - blocking factor (records/block) primary access methods and attributes. x - number of level in multi-level index b I1 - number of first-level index blocks D - number of distinct values sl - fraction of records satisfying equality on attribute. s - selection cardinality (sl*r) (avg. # of records selected.) Some values change, some don t. 3 4 Calculate cost for SELECT operation Cost is measured in block accesses For unique values: S1: Linear search for record (b/2) average case, b - worst case S2: Binary search - log 2 b S3: Primary index or hash : x (number of levels in index) Select for non-unique vals S2a: Binary search - log 2 b + «(s/bfr)» -1 (s is number of duplicate records) S4: ordering index and range of values - x+b/2 (assumes 1/2 of records satisfy relation ) which is average but not correct. S5: clustering index: x + «(s/bfr)» blocks accessed S6: B+ tree: x + s (if using <, <=, etc. x+ b l1 /2 + r/2) 6 and Procedural Abstraction 1

2 Finally, S7: Conjuctions: clauses with ands (use any method) then brute force on end result Example EMPLOYEE(Fname, Minit, Lname, SSN, Bdate, Address, Sex, Salary, Superssn, DNO) r = 10,000 records b = 2000 blocks bfr = Assuming the following access paths 1. Clustering index on salary x salary = 3, sl salary = Secondary index on SSN x ssn = 4 3. Secondary index on DNO x dno = 2 B l1dnum =4, d dno = 125, s dno = Secondary index on SEX, x sex = 1, d sex = 2, s sex = 5000 Compute cost for σ ssn= (EMPLOYEE) Calculate: 1. S1 (linear search) = 2000 (non-key) 1000(key) 2. S6 (secondary index) = x ssn + 1 = 5 since secondary index on SSN x ssn = Compute cost for Compute cost for σ DNO>5 (EMPLOYEE) Calculate: 1. S1 = 2000 (non-key) so need to search all. 2. S6 (secondary index) = x dno + (b l1dno /2)+(r/2) or (4 + (4/2) + (10000/2) = 5004) since secondary index on DNO x dno = 2 B l1dnum =4, d dno = 125, s dno = 80 σ DNO=5 (EMPLOYEE) Calculate: 1. S1 = S6 = x dno + s dno =(2+80) since since secondary index on DNO x dno = 2 B l1dnum =4, d dno = 125, s dno = and Procedural Abstraction 2

3 More complicated example Cont. σ DNO=5 and SALARY>30000 and SEX= F (EMPLOYEE) Pick each and optimize (then find cost of brute force on rest) 1. Do DNO=5 first, cost = Do SALARY>30000 first, cost = x salary + (b/2) = = Do SEX= f first, cost = x sex + s sex = Choose DNO=5 first, lowest cost. Then check salary>30000 and sex = f as records are read into memory Prof. Billibon Yoshimi Computing join costs Need an estimate of size of total join = # selected / Cartesian Product (also called the join selectivity) Use R to denote # of tuples in R, js = R c S / R x S Join costs If optimizer has good estimates for js, it can approximate size of join with js* R * S! Typical Algorithm costs If c is equality relation and c is key of R then Js < S and js [ ( 1 / R ) Cost for Nested Loop join Cost for Single-loop join b R - blocks for R, b S -blocks for S J1: Cost = b R +(b R *b S )+ (js* R * S /bfr RS ) assuming only three buffers. S has an x s -level index. s b is average # of records matching select. For secondary index: Cost = b R + ( R *(x B +s B ))+((js* R * S )/bfr RS ) For clustering index: Cost = b R + ( R *(x B +s B /bfr B )+((js* R * S )/bfr RS ) and Procedural Abstraction 3

4 Single-loop join cont Finally cost of sort-merge join If B is the primary index: Cost = b R + (R * ( x B + 1)) + ((js* R * S )/bfr RS ) If hash key exists for A or B Cost = b R + ( R *h) + ((js* R * S )/bfr RS ) Where h>=1 is average # of blocks required to retrieve record given a hashkey. If files are already sorted then Cost = br + bs + ((js* R * S )/bfr RS ) (worst case since the attribute could be the same for all tuples ) If unsorted (2*b R *(1+log 2 b R ) + (2*b S +(1+log 2 b S )+ above Example from book Optimize this query EMPLOYEE(Fname, Minit, Lname, SSN, Bdate, Address, Sex, Salary, Superssn, DNO) And DEPARTMENT(Dname, Dnumber, Mgrssn, Mgrstartdate) EMPLOYEE = 10,000 b EMPLOYEE =13 DEPARTMENT = 125 b DEPARTMENT = 13 EMPLOYEE Dno=Dnumber DEPARTMENT Primary index on Dnumber for DEPARTMENT. x Dnumber = 1 Secondary index on Mgrssn for DEPARTMENT. x Mgrssn =2 and s Mgrssn =1 Js = (1/ DEPARTMENT ) = 1/125 And bfr ED = 4 records/block Calculations Calculations For nested loop join, 2 cases. 1. Using EMPLOYEE in outer loop. be + (be *bd) + ((js * r E *r D )/bfr ED ) (2000*13) + ((1/125 * 10000*125)/4) 30,500 block accesses 2. Using DEPARTMENT in outer loop (13 * 2000) + ((1/125 * 10000*125)/4) 28,513 block accesses For single loop join, EMPLOYEE as outer loop. b E + (r E * (x Dnumber + 1)) +((js * r E *r D )/bfr ED ) (10,000*(1+1)) + ((1/125 * 10000*125)/4) 24, and Procedural Abstraction 4

5 Calculations Remember secondary index on non key DNO for EMPLOYEE with x Dno = 2 and b I1Dno = 4 and d Dno = 125 and s Dno =80 For single loop join, DEPARTMENT as OL. b D + (r D * (x Dno + S Dno )) +((js * r E *r D )/bfr ED ) 13 + (10,000*(2+80)) + ((1/125 * 10000*125)/4) 12,763 block accesses 25 Points to note Single loop join with DEPARTMENT as outerloop and EMPLOYEE as inter loop makes fewest disk accesses. But, if # of memory buffers allowed for DEPARTMENT went up to 15, ALL of DEPARTMENT could be read into memory! Nested loop join with DEPT as outer loop would be 4513 disk accesses! 26 Multiple relation queries (join ordering) Joins can be manipulated through commutability and associativity rules. Large number of join order permutations. QO - join ordering Put query tree in left-deep order (if a node has a right child, the child is a base relation) Left child is the outer loop, right child is the probe. Left-deep ordering is amenable to pipelining Note for materialization Write this query on board If intermediate tables are generated, optimizer s goal is to minimize size of intermediate tables. SELECT Pnumber, Dnum, Lname, Address, Bdate FROM PROJECT, DEPARTMENT, EMPLOYEE WHERE Dnum=Dnumber AND Mgrssn=SSN AND PLOCATION= Stafford ; and Procedural Abstraction 5

6 What are the joins without CP Given EMPLOYEE(Fname, Minit, Lname, SSN, Bdate, Address, Sex, Salary, Superssn, DNO) DEPARTMENT(Dname, Dnumber, Mgrssn, Mgrstartdate) PROJECT(Pname, Pnumber, Plocation,Dnum) Initial tree looks like (copy to board) π P.Pnumber, P.Dnum, P.Lname, E.Address, E.Bdate P.Plocation = Stafford P P.Dnum=D.Dnumber D.MgrSSN=E.SSN D E Stats about the database Analyze PROJECT DEPARTMENT EMPLOYEE Look at tables, determine join method and access methods. 33 DEPARTMENT has no index. Can only do table scan. Do SELECT on PROJECT table first though! 34 Analyze PROJECT In materialized approach Estimate # of records returned from PROJECT 2000/200 = 10 Requires 2 disk accesses to get index then at most 10 to get record. Table scan (on the other hand takes 100 blocks) Estimate space required for result of Select on PROJECT NUM_ROWS/BLOCKS =bfr for PROJECTlike records. = 20. Average # of records with same location = 2000/200 (10 records) (can fit in 1block) and Procedural Abstraction 6

7 Now calculate first JOIN First join continued PROJECT DEPARTMENT Type of join (only nested) 1 read for temp. 5 reads for DEPARTMENT (total cost 6 blocks) Cost to write out results (total number of matches?) Using key attribute (must be <= temp ) (at most 10 tuples) Assume bfr for temp2 is 5 then takes 2 blocks to write results And so on an so forth Oracle optimization Rule based heuristics Cost-based analysis - using tables like we just did. 39 DB programmer can give optimizer hints. If distributions are not even then indicies can be very useful for narrowing some information. (can also specify access paths, join order, optimization approach, type of joins.) 40 Semantic query optimization Some queries have no valid tuples due to constraints. Select e.fname, e.lname From EMPLOYEE as E, EMPLOYEE as F Where E.superssn = F.ssn and e.salary>f.salary; If we impose constraint that no employee earns more than immediate supervisor. We can reject query outright. (semantics) and Procedural Abstraction 7