Ch. 16: Query Processing and Optimization

Size: px
Start display at page:

Download "Ch. 16: Query Processing and Optimization"

Transcription

1 Ch. 16: Query Processing and Optimization Learning Goals: - Understand the factors affecting response time of a query, & - Concepts behind PLAN_TABLE, EXPLAIN statement Query Processing Steps Strategies for relational operations * Select (Point, Range), Join, set operations * Cost models Query Optimization Module * Input: e.g. Query Trees / Graphs * Output: e.g. Execution plans * Algorithms: e.g. Dynamic Programming

2 2 Query Processing Process is similar to compilation * SQL Query --> [scanner, parser] --> query tree/graph - --> [OPTIMIZER] --> efficient execution plan - --> [code generator] --> code to execute the plan - --> [run-time support] --> query result We will FOCUS on the optimizer module * Output:: Execution plan = set of selected access routines - from possible access routines for each relational operator * Input: Query tree or query graph * Algorithms: Systematic Enumeration/ Dynamic Programming - Algebraic optimization, Other heuristics - Semantic Optimization, Source to Source transforms Access Routines for relational operations & cost models * Select (Point, Range): scan, binary search, index-search * Join: nested loop, sort-merge, hybrid hash

3 : Cost Models for Access Routines Cost Metrics = #block accesses (OR response time in seconds) * focus on I/O cost, not CPU/communication I/O Cost DEPENDS ON * Search criteria: point/range query on ordering /other fields * File Structures: heap, sorted, hashed. * Index-type: primary, clustering, secondary, B+tree, multi-level,... * Other factors, e.g. buffering, disk placement, materialization, - Overflow / free-space mgmt, spanned/unspanned blocking,... Parameters of Cost Models * s = selectivity = fraction of tuples selected = result / table * S(field) = selection cardinality = r(d) / (#distinct values of field) * B, bd, bi(j) = block size, #blocks in datafile/ at level j of index * r(d) = #records in data file * u, bfr(d), bfr(i) = utilization, blocking factors for datafile/index * fo, d = fanout and depth of index * R, P, Pr = record size, tree pointersize, data record pointer size * e = Pr[a record in overflow area] * E[overflow chain length]

4 4 16.1: Access Routines for Relational Operators Access routine - is an execution strategy for a relational operator * May use available file-structures and indexes * Specifies algorithms, and applicability Point Query: (i) Linear, (ii) Binary, (iii) Hash, (iv) Indexed search Range Query: (1) Linear search, (2) Binary search then scan - 3. Find 1st record in, then scan datafile (Primary/Clustered index) - 4. Find 1st record in, then scan index leafs (Secondary index/b+tree) Conjunctive selection - 1. Use composite index - 2. Process one condition first followed by scan of result - 3. Intersect (record pointers retrieved by different conditions) Ex. Provide algebraic formula for access routines * Point Search (UNIQUE field): bd/2 ; log(bd); 1+ e; d + 1; - not unique => + S/bfr(d) to sorted/hashed file; bd for heap * Range Search: bd or bd/2 + s*bd; log(bd) + s*bd; d s*bd; - secondary index : d + s*[bi(leaf) + r(d)] worst case Ex. Algebraic formula for a query Q3 from QP-1 (Table 1)

5 : Implementing JOIN operation + Cost Focus on two way equi-join:: join(r, S, R.A = S.B) Join-selectivity= js(r.a=s.b)= join result /( R * S ); X =#records in X * => Join result has b(rs)= (js* R * S / bfr(rs)) blocks (1)Nested loop: For each block in R, (a) Read matching record in S - using scan or indexed search, and (b) Write partial result * I/O Cost = br + (br * bs) + b(rs) - Indexed search cost= f(available index(s.b)), see Page = br + [ R *(d(s.b) + S(S.B))] + b(rs), secondary index(s.b) (2) Sort-Merge: (i) If Needed, Sort R on R.A, Sort S on S.B - (ii) Scan R and S block by block [See Fig. 16.2(a), pp 498] * Very efficient if R and S already sorted, Cost = br + bs + b(rs) * Sorting => Add k*{br * log(br) + bs * log(bs)} (3) Hash: (i) hash records of R, S into a common hash file (CHF) - (ii) scan CHF to compute and write results * Cost= 3*(bR+bS) + b(rs), w/ buffering (hybrid hash, pp501)

6 6 Dominance Analysis of Access Routines Ex. Compare scan and secondary-index for Range queries? * Heap file Scan: bd; <= Secondary index: d + s*[bi(leaf) + r(d)] - OR selectivity(s) >= (bd-d) / [bi(leaf)+rd] > bd/rd = (1/bfr(d)) * example: bfr(d) = 10; selectivity = 0.15 ; Ex.Nested loop: Which relation should be in outer loop? * Smaller one => reduces cost Q? Which strategy wins for join(r,s), IF (br > bs) and * (1) primary index on R.A in R, S.B in S (depth = x) - Let #record/distinct value of join field = a - (br = bs) => sort merge if br = bs - (br > a*x*bs) => nested loop with index-scan * (2) Heap Files, br = bs :: hash-join (log(br)>10) * (3) Hashed Files R on R.A, S on S.B, - (br = bs) => hash join if br = bs, log(br) > 10 - (br > 4*bfr(inner loop)*bs) => nested loop with index-scan? - see Q6, row 2 in QP-2, parametrize (br/bs)

7 7 Dominance Analysis of Access Routines (Contd.) Q? Which strategy wins for join(r,s), IF (br > bs) and * (4) primary index on R.A, S not sorted on primary key S.B, - bs = br, bfr(r) = 3, index-depth = 3, (Hint: unique matches) - Nested loop (inner = indexed search on R) cost = 4bR + b(rs) * (5)bS < memory << br ; cost = br + bs + B(RS) for all 3-1. buffer + index/sort/hash S in memory; - 2. For each block in R, bring the block and write result Q? Which strategies can be used for non-equijoins? - for select-project-equijoin? Summary comparison of join strategies * Nested loop is a dark horse, watch out! - e.g. (br/bs) >> 1, or bfr >> 1 - OR only one file has good search structure * Both files sorted, sort-merge can win - unless (br/bs) >> 1, or bfr >> 1 * Both files have No good search structure, Hybrid hash can win - unless (br/bs) >> 1, or bfr >> 1

8 , 16.4: Query Optimization Module Output: Sequence or Set of access routines Input:(i) Query Tree: syntax tree for eqv. rel. algebra query - Leafs = input relations :: Non-leafs = rel. algebra operations * Initial canonical tree: Easy to generate from parser (Fig. 16.4(a)) - cartesian product ( ) of all rel.+ 1 σ + 1 π - Ex. Fig. 16.3(a) & (b) pp 505 with Q2 (pp 504) (ii)query graph: constraints in tuple calculus exp. (Fig. 16.5/pp 512) * Nodes = relations (tuple variables) OR constants (double circle) - Edges = join conditions * We will not use it. Came from Ingres/QUEL optimizer. ALGORITHM 1: Source to Source Transform (Ch 16.4) * Semantic transformation rules, e.g. integrity constraints (IC) - Ex. Q(pp 522): Find Emps who earn more than their supervisors - IC: No employee earns more than his/her supervisor. * Implemented by Thm. prover => exponential in #applicable rules * Alternative is hand recoding (See Add. Ex. grad-2)

9 ! 9 Query Optimization Algorithms (Contd.) ALGORITHM 2: Algebraic optimization by transforming query trees * Start w/ Initial canonical tree * Use laws of relational algebra (Ex. rule 4, pp 509) * General steps (pp 511) towards applying those operations first - which reduce the size of intermediate results Ex. Trace: Fig (pp 507-9) * Fig. 16.4(b) move σs down the tree (close to table) * Fig. 16.4(c) more restrictive σ 1st (pname = Aquarium ) - commute cartesian products, rearrange σ conditions * Fig. 16.4(d): Combine (, σ) into joins * Fig. 16.4(e): Move πs down the tree and add π - to drop fields not needed for result (and join/selects) ALGORITHM 3: Dynamic Programming w/ cost models * 1. Choose an ordering of join operation (e.g. left to right) * 2. Given ordering, enumerate combination of access routines - choose low-cost combination, 1 access routine per rel. op. * Used in commercial systems, Step 2 is cubic in #relations, - Step 1. is exponential and optimizers scrimp here!

Chapter 13: Query Processing. Basic Steps in Query Processing

Chapter 13: Query Processing. Basic Steps in Query Processing Chapter 13: Query Processing! Overview! Measures of Query Cost! Selection Operation! Sorting! Join Operation! Other Operations! Evaluation of Expressions 13.1 Basic Steps in Query Processing 1. Parsing

More information

Comp 5311 Database Management Systems. 16. Review 2 (Physical Level)

Comp 5311 Database Management Systems. 16. Review 2 (Physical Level) Comp 5311 Database Management Systems 16. Review 2 (Physical Level) 1 Main Topics Indexing Join Algorithms Query Processing and Optimization Transactions and Concurrency Control 2 Indexing Used for faster

More information

SQL Query Evaluation. Winter 2006-2007 Lecture 23

SQL Query Evaluation. Winter 2006-2007 Lecture 23 SQL Query Evaluation Winter 2006-2007 Lecture 23 SQL Query Processing Databases go through three steps: Parse SQL into an execution plan Optimize the execution plan Evaluate the optimized plan Execution

More information

Query Processing C H A P T E R12. Practice Exercises

Query Processing C H A P T E R12. Practice Exercises C H A P T E R12 Query Processing Practice Exercises 12.1 Assume (for simplicity in this exercise) that only one tuple fits in a block and memory holds at most 3 blocks. Show the runs created on each pass

More information

Elena Baralis, Silvia Chiusano Politecnico di Torino. Pag. 1. Physical Design. Phases of database design. Physical design: Inputs.

Elena Baralis, Silvia Chiusano Politecnico di Torino. Pag. 1. Physical Design. Phases of database design. Physical design: Inputs. Phases of database design Application requirements Conceptual design Database Management Systems Conceptual schema Logical design ER or UML Physical Design Relational tables Logical schema Physical design

More information

Databases and Information Systems 1 Part 3: Storage Structures and Indices

Databases and Information Systems 1 Part 3: Storage Structures and Indices bases and Information Systems 1 Part 3: Storage Structures and Indices Prof. Dr. Stefan Böttcher Fakultät EIM, Institut für Informatik Universität Paderborn WS 2009 / 2010 Contents: - database buffer -

More information

DATABASE DESIGN - 1DL400

DATABASE DESIGN - 1DL400 DATABASE DESIGN - 1DL400 Spring 2015 A course on modern database systems!! http://www.it.uu.se/research/group/udbl/kurser/dbii_vt15/ Kjell Orsborn! Uppsala Database Laboratory! Department of Information

More information

Chapter 8: Structures for Files. Truong Quynh Chi tqchi@cse.hcmut.edu.vn. Spring- 2013

Chapter 8: Structures for Files. Truong Quynh Chi tqchi@cse.hcmut.edu.vn. Spring- 2013 Chapter 8: Data Storage, Indexing Structures for Files Truong Quynh Chi tqchi@cse.hcmut.edu.vn Spring- 2013 Overview of Database Design Process 2 Outline Data Storage Disk Storage Devices Files of Records

More information

Oracle Database 11g: SQL Tuning Workshop

Oracle Database 11g: SQL Tuning Workshop Oracle University Contact Us: + 38516306373 Oracle Database 11g: SQL Tuning Workshop Duration: 3 Days What you will learn This Oracle Database 11g: SQL Tuning Workshop Release 2 training assists database

More information

Lecture 1: Data Storage & Index

Lecture 1: Data Storage & Index Lecture 1: Data Storage & Index R&G Chapter 8-11 Concurrency control Query Execution and Optimization Relational Operators File & Access Methods Buffer Management Disk Space Management Recovery Manager

More information

www.gr8ambitionz.com

www.gr8ambitionz.com Data Base Management Systems (DBMS) Study Material (Objective Type questions with Answers) Shared by Akhil Arora Powered by www. your A to Z competitive exam guide Database Objective type questions Q.1

More information

Oracle Database 11g: SQL Tuning Workshop Release 2

Oracle Database 11g: SQL Tuning Workshop Release 2 Oracle University Contact Us: 1 800 005 453 Oracle Database 11g: SQL Tuning Workshop Release 2 Duration: 3 Days What you will learn This course assists database developers, DBAs, and SQL developers to

More information

University of Massachusetts Amherst Department of Computer Science Prof. Yanlei Diao

University of Massachusetts Amherst Department of Computer Science Prof. Yanlei Diao University of Massachusetts Amherst Department of Computer Science Prof. Yanlei Diao CMPSCI 445 Midterm Practice Questions NAME: LOGIN: Write all of your answers directly on this paper. Be sure to clearly

More information

CIS 631 Database Management Systems Sample Final Exam

CIS 631 Database Management Systems Sample Final Exam CIS 631 Database Management Systems Sample Final Exam 1. (25 points) Match the items from the left column with those in the right and place the letters in the empty slots. k 1. Single-level index files

More information

Elena Baralis, Silvia Chiusano Politecnico di Torino. Pag. 1. Query optimization. DBMS Architecture. Query optimizer. Query optimizer.

Elena Baralis, Silvia Chiusano Politecnico di Torino. Pag. 1. Query optimization. DBMS Architecture. Query optimizer. Query optimizer. DBMS Architecture INSTRUCTION OPTIMIZER Database Management Systems MANAGEMENT OF ACCESS METHODS BUFFER MANAGER CONCURRENCY CONTROL RELIABILITY MANAGEMENT Index Files Data Files System Catalog BASE It

More information

Physical Data Organization

Physical Data Organization Physical Data Organization Database design using logical model of the database - appropriate level for users to focus on - user independence from implementation details Performance - other major factor

More information

Query Optimization for Distributed Database Systems Robert Taylor Candidate Number : 933597 Hertford College Supervisor: Dr.

Query Optimization for Distributed Database Systems Robert Taylor Candidate Number : 933597 Hertford College Supervisor: Dr. Query Optimization for Distributed Database Systems Robert Taylor Candidate Number : 933597 Hertford College Supervisor: Dr. Dan Olteanu Submitted as part of Master of Computer Science Computing Laboratory

More information

Unit 4.3 - Storage Structures 1. Storage Structures. Unit 4.3

Unit 4.3 - Storage Structures 1. Storage Structures. Unit 4.3 Storage Structures Unit 4.3 Unit 4.3 - Storage Structures 1 The Physical Store Storage Capacity Medium Transfer Rate Seek Time Main Memory 800 MB/s 500 MB Instant Hard Drive 10 MB/s 120 GB 10 ms CD-ROM

More information

CSE 326, Data Structures. Sample Final Exam. Problem Max Points Score 1 14 (2x7) 2 18 (3x6) 3 4 4 7 5 9 6 16 7 8 8 4 9 8 10 4 Total 92.

CSE 326, Data Structures. Sample Final Exam. Problem Max Points Score 1 14 (2x7) 2 18 (3x6) 3 4 4 7 5 9 6 16 7 8 8 4 9 8 10 4 Total 92. Name: Email ID: CSE 326, Data Structures Section: Sample Final Exam Instructions: The exam is closed book, closed notes. Unless otherwise stated, N denotes the number of elements in the data structure

More information

Datenbanksysteme II: Implementation of Database Systems Implementing Joins

Datenbanksysteme II: Implementation of Database Systems Implementing Joins Datenbanksysteme II: Implementation of Database Systems Implementing Joins Material von Prof. Johann Christoph Freytag Prof. Kai-Uwe Sattler Prof. Alfons Kemper, Dr. Eickler Prof. Hector Garcia-Molina

More information

Overview of Storage and Indexing

Overview of Storage and Indexing Overview of Storage and Indexing Chapter 8 How index-learning turns no student pale Yet holds the eel of science by the tail. -- Alexander Pope (1688-1744) Database Management Systems 3ed, R. Ramakrishnan

More information

Symbol Tables. Introduction

Symbol Tables. Introduction Symbol Tables Introduction A compiler needs to collect and use information about the names appearing in the source program. This information is entered into a data structure called a symbol table. The

More information

University of Aarhus. Databases 2009. 2009 IBM Corporation

University of Aarhus. Databases 2009. 2009 IBM Corporation University of Aarhus Databases 2009 Kirsten Ann Larsen What is good performance? Elapsed time End-to-end In DB2 Resource consumption CPU I/O Memory Locks Elapsed time = Sync. I/O + CPU + wait time I/O

More information

Inside the PostgreSQL Query Optimizer

Inside the PostgreSQL Query Optimizer Inside the PostgreSQL Query Optimizer Neil Conway neilc@samurai.com Fujitsu Australia Software Technology PostgreSQL Query Optimizer Internals p. 1 Outline Introduction to query optimization Outline of

More information

Evaluation of Expressions

Evaluation of Expressions Query Optimization Evaluation of Expressions Materialization: one operation at a time, materialize intermediate results for subsequent use Good for all situations Sum of costs of individual operations

More information

Guide to Performance and Tuning: Query Performance and Sampled Selectivity

Guide to Performance and Tuning: Query Performance and Sampled Selectivity Guide to Performance and Tuning: Query Performance and Sampled Selectivity A feature of Oracle Rdb By Claude Proteau Oracle Rdb Relational Technology Group Oracle Corporation 1 Oracle Rdb Journal Sampled

More information

Record Storage and Primary File Organization

Record Storage and Primary File Organization Record Storage and Primary File Organization 1 C H A P T E R 4 Contents Introduction Secondary Storage Devices Buffering of Blocks Placing File Records on Disk Operations on Files Files of Unordered Records

More information

Data Warehousing und Data Mining

Data Warehousing und Data Mining Data Warehousing und Data Mining Multidimensionale Indexstrukturen Ulf Leser Wissensmanagement in der Bioinformatik Content of this Lecture Multidimensional Indexing Grid-Files Kd-trees Ulf Leser: Data

More information

Overview of Storage and Indexing. Data on External Storage. Alternative File Organizations. Chapter 8

Overview of Storage and Indexing. Data on External Storage. Alternative File Organizations. Chapter 8 Overview of Storage and Indexing Chapter 8 How index-learning turns no student pale Yet holds the eel of science by the tail. -- Alexander Pope (1688-1744) Database Management Systems 3ed, R. Ramakrishnan

More information

Advanced Oracle SQL Tuning

Advanced Oracle SQL Tuning Advanced Oracle SQL Tuning Seminar content technical details 1) Understanding Execution Plans In this part you will learn how exactly Oracle executes SQL execution plans. Instead of describing on PowerPoint

More information

Storage and File Structure

Storage and File Structure Storage and File Structure Chapter 10: Storage and File Structure Overview of Physical Storage Media Magnetic Disks RAID Tertiary Storage Storage Access File Organization Organization of Records in Files

More information

Query tuning by eliminating throwaway

Query tuning by eliminating throwaway Query tuning by eliminating throwaway This paper deals with optimizing non optimal performing queries. Abstract Martin Berg (martin.berg@oracle.com) Server Technology System Management & Performance Oracle

More information

Copyright 2007 Ramez Elmasri and Shamkant B. Navathe. Slide 13-1

Copyright 2007 Ramez Elmasri and Shamkant B. Navathe. Slide 13-1 Slide 13-1 Chapter 13 Disk Storage, Basic File Structures, and Hashing Chapter Outline Disk Storage Devices Files of Records Operations on Files Unordered Files Ordered Files Hashed Files Dynamic and Extendible

More information

Partitioning under the hood in MySQL 5.5

Partitioning under the hood in MySQL 5.5 Partitioning under the hood in MySQL 5.5 Mattias Jonsson, Partitioning developer Mikael Ronström, Partitioning author Who are we? Mikael is a founder of the technology behind NDB

More information

Access Path Selection in a Relational Database Management System

Access Path Selection in a Relational Database Management System Access Path Selection in a Relational Database Management System P. Griffiths Selinger M. M. Astrahan D. D. Chamberlin R. A. Lorie T. G. Price IBM Research Division, San Jose, California 95193 ABSTRACT:

More information

Krishna Institute of Engineering & Technology, Ghaziabad Department of Computer Application MCA-213 : DATA STRUCTURES USING C

Krishna Institute of Engineering & Technology, Ghaziabad Department of Computer Application MCA-213 : DATA STRUCTURES USING C Tutorial#1 Q 1:- Explain the terms data, elementary item, entity, primary key, domain, attribute and information? Also give examples in support of your answer? Q 2:- What is a Data Type? Differentiate

More information

Chapter 13. Disk Storage, Basic File Structures, and Hashing

Chapter 13. Disk Storage, Basic File Structures, and Hashing Chapter 13 Disk Storage, Basic File Structures, and Hashing Chapter Outline Disk Storage Devices Files of Records Operations on Files Unordered Files Ordered Files Hashed Files Dynamic and Extendible Hashing

More information

Chapter 5: Overview of Query Processing

Chapter 5: Overview of Query Processing Chapter 5: Overview of Query Processing Query Processing Overview Query Optimization Distributed Query Processing Steps Acknowledgements: I am indebted to Arturas Mazeika for providing me his slides of

More information

INTRODUCTION The collection of data that makes up a computerized database must be stored physically on some computer storage medium.

INTRODUCTION The collection of data that makes up a computerized database must be stored physically on some computer storage medium. Chapter 4: Record Storage and Primary File Organization 1 Record Storage and Primary File Organization INTRODUCTION The collection of data that makes up a computerized database must be stored physically

More information

Query Processing. Q Query Plan. Example: Select B,D From R,S Where R.A = c S.E = 2 R.C=S.C. Himanshu Gupta CSE 532 Query Proc. 1

Query Processing. Q Query Plan. Example: Select B,D From R,S Where R.A = c S.E = 2 R.C=S.C. Himanshu Gupta CSE 532 Query Proc. 1 Q Query Plan Query Processing Example: Select B,D From R,S Where R.A = c S.E = 2 R.C=S.C Himanshu Gupta CSE 532 Query Proc. 1 How do we execute query? One idea - Do Cartesian product - Select tuples -

More information

Chapter 13 Disk Storage, Basic File Structures, and Hashing.

Chapter 13 Disk Storage, Basic File Structures, and Hashing. Chapter 13 Disk Storage, Basic File Structures, and Hashing. Copyright 2004 Pearson Education, Inc. Chapter Outline Disk Storage Devices Files of Records Operations on Files Unordered Files Ordered Files

More information

SQL Server Query Tuning

SQL Server Query Tuning SQL Server Query Tuning Klaus Aschenbrenner Independent SQL Server Consultant SQLpassion.at Twitter: @Aschenbrenner About me Independent SQL Server Consultant International Speaker, Author Pro SQL Server

More information

Operating Systems CSE 410, Spring 2004. File Management. Stephen Wagner Michigan State University

Operating Systems CSE 410, Spring 2004. File Management. Stephen Wagner Michigan State University Operating Systems CSE 410, Spring 2004 File Management Stephen Wagner Michigan State University File Management File management system has traditionally been considered part of the operating system. Applications

More information

DATABASDESIGN FÖR INGENJÖRER - 1DL124

DATABASDESIGN FÖR INGENJÖRER - 1DL124 1 DATABASDESIGN FÖR INGENJÖRER - 1DL124 Sommar 2005 En introduktionskurs i databassystem http://user.it.uu.se/~udbl/dbt-sommar05/ alt. http://www.it.uu.se/edu/course/homepage/dbdesign/st05/ Kjell Orsborn

More information

In-Memory Database: Query Optimisation. S S Kausik (110050003) Aamod Kore (110050004) Mehul Goyal (110050017) Nisheeth Lahoti (110050027)

In-Memory Database: Query Optimisation. S S Kausik (110050003) Aamod Kore (110050004) Mehul Goyal (110050017) Nisheeth Lahoti (110050027) In-Memory Database: Query Optimisation S S Kausik (110050003) Aamod Kore (110050004) Mehul Goyal (110050017) Nisheeth Lahoti (110050027) Introduction Basic Idea Database Design Data Types Indexing Query

More information

Performance Evaluation of Natural and Surrogate Key Database Architectures

Performance Evaluation of Natural and Surrogate Key Database Architectures Performance Evaluation of Natural and Surrogate Key Database Architectures Sebastian Link 1, Ivan Luković 2, Pavle ogin *)1 1 Victoria University of Wellington, Wellington, P.O. Box 600, New Zealand sebastian.link@vuw.ac.nz

More information

File Management. File Management

File Management. File Management File Management 1 File Management File management system consists of system utility programs that run as privileged applications Input to applications is by means of a file Output is saved in a file for

More information

Question 1. Relational Data Model [17 marks] Question 2. SQL and Relational Algebra [31 marks]

Question 1. Relational Data Model [17 marks] Question 2. SQL and Relational Algebra [31 marks] EXAMINATIONS 2005 MID-YEAR COMP 302 Database Systems Time allowed: Instructions: 3 Hours Answer all questions. Make sure that your answers are clear and to the point. Write your answers in the spaces provided.

More information

IBM DB2: LUW Performance Tuning and Monitoring for Single and Multiple Partition DBs

IBM DB2: LUW Performance Tuning and Monitoring for Single and Multiple Partition DBs coursemonster.com/au IBM DB2: LUW Performance Tuning and Monitoring for Single and Multiple Partition DBs View training dates» Overview Learn how to tune for optimum performance the IBM DB2 9 for Linux,

More information

DB2 for Linux, UNIX, and Windows Performance Tuning and Monitoring Workshop

DB2 for Linux, UNIX, and Windows Performance Tuning and Monitoring Workshop DB2 for Linux, UNIX, and Windows Performance Tuning and Monitoring Workshop Duration: 4 Days What you will learn Learn how to tune for optimum performance the IBM DB2 9 for Linux, UNIX, and Windows relational

More information

Understanding Query Processing and Query Plans in SQL Server. Craig Freedman Software Design Engineer Microsoft SQL Server

Understanding Query Processing and Query Plans in SQL Server. Craig Freedman Software Design Engineer Microsoft SQL Server Understanding Query Processing and Query Plans in SQL Server Craig Freedman Software Design Engineer Microsoft SQL Server Outline SQL Server engine architecture Query execution overview Showplan Common

More information

Unique column combinations

Unique column combinations Unique column combinations Arvid Heise Guest lecture in Data Profiling and Data Cleansing Prof. Dr. Felix Naumann Agenda 2 Introduction and problem statement Unique column combinations Exponential search

More information

Efficiently Identifying Inclusion Dependencies in RDBMS

Efficiently Identifying Inclusion Dependencies in RDBMS Efficiently Identifying Inclusion Dependencies in RDBMS Jana Bauckmann Department for Computer Science, Humboldt-Universität zu Berlin Rudower Chaussee 25, 12489 Berlin, Germany bauckmann@informatik.hu-berlin.de

More information

DATA STRUCTURES USING C

DATA STRUCTURES USING C DATA STRUCTURES USING C QUESTION BANK UNIT I 1. Define data. 2. Define Entity. 3. Define information. 4. Define Array. 5. Define data structure. 6. Give any two applications of data structures. 7. Give

More information

Answer Key. UNIVERSITY OF CALIFORNIA College of Engineering Department of EECS, Computer Science Division

Answer Key. UNIVERSITY OF CALIFORNIA College of Engineering Department of EECS, Computer Science Division Answer Key UNIVERSITY OF CALIFORNIA College of Engineering Department of EECS, Computer Science Division CS186 Fall 2003 Eben Haber Midterm Midterm Exam: Introduction to Database Systems This exam has

More information

Query Optimization in a Memory-Resident Domain Relational Calculus Database System

Query Optimization in a Memory-Resident Domain Relational Calculus Database System Query Optimization in a Memory-Resident Domain Relational Calculus Database System KYU-YOUNG WHANG and RAVI KRISHNAMURTHY IBM Thomas J. Watson Research Center We present techniques for optimizing queries

More information

Relational Databases

Relational Databases Relational Databases Jan Chomicki University at Buffalo Jan Chomicki () Relational databases 1 / 18 Relational data model Domain domain: predefined set of atomic values: integers, strings,... every attribute

More information

5. A full binary tree with n leaves contains [A] n nodes. [B] log n 2 nodes. [C] 2n 1 nodes. [D] n 2 nodes.

5. A full binary tree with n leaves contains [A] n nodes. [B] log n 2 nodes. [C] 2n 1 nodes. [D] n 2 nodes. 1. The advantage of.. is that they solve the problem if sequential storage representation. But disadvantage in that is they are sequential lists. [A] Lists [B] Linked Lists [A] Trees [A] Queues 2. The

More information

File Management. Chapter 12

File Management. Chapter 12 File Management Chapter 12 File Management File management system is considered part of the operating system Input to applications is by means of a file Output is saved in a file for long-term storage

More information

Chapter 13: Query Optimization

Chapter 13: Query Optimization Chapter 13: Query Optimization Database System Concepts, 6 th Ed. See www.db-book.com for conditions on re-use Chapter 13: Query Optimization Introduction Transformation of Relational Expressions Catalog

More information

Chapter 13. Chapter Outline. Disk Storage, Basic File Structures, and Hashing

Chapter 13. Chapter Outline. Disk Storage, Basic File Structures, and Hashing Chapter 13 Disk Storage, Basic File Structures, and Hashing Copyright 2007 Ramez Elmasri and Shamkant B. Navathe Chapter Outline Disk Storage Devices Files of Records Operations on Files Unordered Files

More information

Big Data and Scripting. Part 4: Memory Hierarchies

Big Data and Scripting. Part 4: Memory Hierarchies 1, Big Data and Scripting Part 4: Memory Hierarchies 2, Model and Definitions memory size: M machine words total storage (on disk) of N elements (N is very large) disk size unlimited (for our considerations)

More information

ENHANCEMENTS TO SQL SERVER COLUMN STORES. Anuhya Mallempati #2610771

ENHANCEMENTS TO SQL SERVER COLUMN STORES. Anuhya Mallempati #2610771 ENHANCEMENTS TO SQL SERVER COLUMN STORES Anuhya Mallempati #2610771 CONTENTS Abstract Introduction Column store indexes Batch mode processing Other Enhancements Conclusion ABSTRACT SQL server introduced

More information

LiTH, Tekniska högskolan vid Linköpings universitet 1(7) IDA, Institutionen för datavetenskap Juha Takkinen 2007-05-24

LiTH, Tekniska högskolan vid Linköpings universitet 1(7) IDA, Institutionen för datavetenskap Juha Takkinen 2007-05-24 LiTH, Tekniska högskolan vid Linköpings universitet 1(7) IDA, Institutionen för datavetenskap Juha Takkinen 2007-05-24 1. A database schema is a. the state of the db b. a description of the db using a

More information

Lecture 6: Query optimization, query tuning. Rasmus Pagh

Lecture 6: Query optimization, query tuning. Rasmus Pagh Lecture 6: Query optimization, query tuning Rasmus Pagh 1 Today s lecture Only one session (10-13) Query optimization: Overview of query evaluation Estimating sizes of intermediate results A typical query

More information

Chapter 2 Data Storage

Chapter 2 Data Storage Chapter 2 22 CHAPTER 2. DATA STORAGE 2.1. THE MEMORY HIERARCHY 23 26 CHAPTER 2. DATA STORAGE main memory, yet is essentially random-access, with relatively small differences Figure 2.4: A typical

More information

MapReduce examples. CSE 344 section 8 worksheet. May 19, 2011

MapReduce examples. CSE 344 section 8 worksheet. May 19, 2011 MapReduce examples CSE 344 section 8 worksheet May 19, 2011 In today s section, we will be covering some more examples of using MapReduce to implement relational queries. Recall how MapReduce works from

More information

ML for the Working Programmer

ML for the Working Programmer ML for the Working Programmer 2nd edition Lawrence C. Paulson University of Cambridge CAMBRIDGE UNIVERSITY PRESS CONTENTS Preface to the Second Edition Preface xiii xv 1 Standard ML 1 Functional Programming

More information

Execution Plans: The Secret to Query Tuning Success. MagicPASS January 2015

Execution Plans: The Secret to Query Tuning Success. MagicPASS January 2015 Execution Plans: The Secret to Query Tuning Success MagicPASS January 2015 Jes Schultz Borland plan? The following steps are being taken Parsing Compiling Optimizing In the optimizing phase An execution

More information

Data storage Tree indexes

Data storage Tree indexes Data storage Tree indexes Rasmus Pagh February 7 lecture 1 Access paths For many database queries and updates, only a small fraction of the data needs to be accessed. Extreme examples are looking or updating

More information

Why Query Optimization? Access Path Selection in a Relational Database Management System. How to come up with the right query plan?

Why Query Optimization? Access Path Selection in a Relational Database Management System. How to come up with the right query plan? Why Query Optimization? Access Path Selection in a Relational Database Management System P. Selinger, M. Astrahan, D. Chamberlin, R. Lorie, T. Price Peyman Talebifard Queries must be executed and execution

More information

D B M G Data Base and Data Mining Group of Politecnico di Torino

D B M G Data Base and Data Mining Group of Politecnico di Torino Database Management Data Base and Data Mining Group of tania.cerquitelli@polito.it A.A. 2014-2015 Optimizer objective A SQL statement can be executed in many different ways The query optimizer determines

More information

1. The memory address of the first element of an array is called A. floor address B. foundation addressc. first address D.

1. The memory address of the first element of an array is called A. floor address B. foundation addressc. first address D. 1. The memory address of the first element of an array is called A. floor address B. foundation addressc. first address D. base address 2. The memory address of fifth element of an array can be calculated

More information

Run$me Query Op$miza$on

Run$me Query Op$miza$on Run$me Query Op$miza$on Robust Op$miza$on for Graphs 2006-2014 All Rights Reserved 1 RDF Join Order Op$miza$on Typical approach Assign es$mated cardinality to each triple pabern. Bigdata uses the fast

More information

External Sorting. Chapter 13. Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1

External Sorting. Chapter 13. Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1 External Sorting Chapter 13 Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1 Why Sort? A classic problem in computer science! Data requested in sorted order e.g., find students in increasing

More information

Oracle DBA Course Contents

Oracle DBA Course Contents Oracle DBA Course Contents Overview of Oracle DBA tasks: Oracle as a flexible, complex & robust RDBMS The evolution of hardware and the relation to Oracle Different DBA job roles(vp of DBA, developer DBA,production

More information

Storage in Database Systems. CMPSCI 445 Fall 2010

Storage in Database Systems. CMPSCI 445 Fall 2010 Storage in Database Systems CMPSCI 445 Fall 2010 1 Storage Topics Architecture and Overview Disks Buffer management Files of records 2 DBMS Architecture Query Parser Query Rewriter Query Optimizer Query

More information

Chapter 9 Joining Data from Multiple Tables. Oracle 10g: SQL

Chapter 9 Joining Data from Multiple Tables. Oracle 10g: SQL Chapter 9 Joining Data from Multiple Tables Oracle 10g: SQL Objectives Identify a Cartesian join Create an equality join using the WHERE clause Create an equality join using the JOIN keyword Create a non-equality

More information

Chapter 6: Physical Database Design and Performance. Database Development Process. Physical Design Process. Physical Database Design

Chapter 6: Physical Database Design and Performance. Database Development Process. Physical Design Process. Physical Database Design Chapter 6: Physical Database Design and Performance Modern Database Management 6 th Edition Jeffrey A. Hoffer, Mary B. Prescott, Fred R. McFadden Robert C. Nickerson ISYS 464 Spring 2003 Topic 23 Database

More information

Efficient Computation of Multiple Group By Queries Zhimin Chen Vivek Narasayya

Efficient Computation of Multiple Group By Queries Zhimin Chen Vivek Narasayya Efficient Computation of Multiple Group By Queries Zhimin Chen Vivek Narasayya Microsoft Research {zmchen, viveknar}@microsoft.com ABSTRACT Data analysts need to understand the quality of data in the warehouse.

More information

Oracle EXAM - 1Z0-117. Oracle Database 11g Release 2: SQL Tuning. Buy Full Product. http://www.examskey.com/1z0-117.html

Oracle EXAM - 1Z0-117. Oracle Database 11g Release 2: SQL Tuning. Buy Full Product. http://www.examskey.com/1z0-117.html Oracle EXAM - 1Z0-117 Oracle Database 11g Release 2: SQL Tuning Buy Full Product http://www.examskey.com/1z0-117.html Examskey Oracle 1Z0-117 exam demo product is here for you to test the quality of the

More information

Understanding SQL Server Execution Plans. Klaus Aschenbrenner Independent SQL Server Consultant SQLpassion.at Twitter: @Aschenbrenner

Understanding SQL Server Execution Plans. Klaus Aschenbrenner Independent SQL Server Consultant SQLpassion.at Twitter: @Aschenbrenner Understanding SQL Server Execution Plans Klaus Aschenbrenner Independent SQL Server Consultant SQLpassion.at Twitter: @Aschenbrenner About me Independent SQL Server Consultant International Speaker, Author

More information

Efficient Iceberg Query Evaluation for Structured Data using Bitmap Indices

Efficient Iceberg Query Evaluation for Structured Data using Bitmap Indices Proc. of Int. Conf. on Advances in Computer Science, AETACS Efficient Iceberg Query Evaluation for Structured Data using Bitmap Indices Ms.Archana G.Narawade a, Mrs.Vaishali Kolhe b a PG student, D.Y.Patil

More information

High-performance XML Storage/Retrieval System

High-performance XML Storage/Retrieval System UDC 00.5:68.3 High-performance XML Storage/Retrieval System VYasuo Yamane VNobuyuki Igata VIsao Namba (Manuscript received August 8, 000) This paper describes a system that integrates full-text searching

More information

1Z0-117 Oracle Database 11g Release 2: SQL Tuning. Oracle

1Z0-117 Oracle Database 11g Release 2: SQL Tuning. Oracle 1Z0-117 Oracle Database 11g Release 2: SQL Tuning Oracle To purchase Full version of Practice exam click below; http://www.certshome.com/1z0-117-practice-test.html FOR Oracle 1Z0-117 Exam Candidates We

More information

a presentation by Kirk Paul Lafler SAS Consultant, Author, and Trainer E-mail: KirkLafler@cs.com

a presentation by Kirk Paul Lafler SAS Consultant, Author, and Trainer E-mail: KirkLafler@cs.com a presentation by Kirk Paul Lafler SAS Consultant, Author, and Trainer E-mail: KirkLafler@cs.com 1 Copyright Kirk Paul Lafler, 1992-2010. All rights reserved. SAS is the registered trademark of SAS Institute

More information

Chapter 14: Query Optimization

Chapter 14: Query Optimization Chapter 14: Query Optimization Database System Concepts 5 th Ed. See www.db-book.com for conditions on re-use Chapter 14: Query Optimization Introduction Transformation of Relational Expressions Catalog

More information

DB2 LUW Performance Tuning and Monitoring for Single and Multiple Partition DBs

DB2 LUW Performance Tuning and Monitoring for Single and Multiple Partition DBs Kod szkolenia: Tytuł szkolenia: CL442PL DB2 LUW Performance Tuning and Monitoring for Single and Multiple Partition DBs Dni: 5 Opis: Learn how to tune for optimum the IBM DB2 9 for Linux, UNIX, and Windows

More information

External Sorting. Why Sort? 2-Way Sort: Requires 3 Buffers. Chapter 13

External Sorting. Why Sort? 2-Way Sort: Requires 3 Buffers. Chapter 13 External Sorting Chapter 13 Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1 Why Sort? A classic problem in computer science! Data requested in sorted order e.g., find students in increasing

More information

Previous Lectures. B-Trees. External storage. Two types of memory. B-trees. Main principles

Previous Lectures. B-Trees. External storage. Two types of memory. B-trees. Main principles B-Trees Algorithms and data structures for external memory as opposed to the main memory B-Trees Previous Lectures Height balanced binary search trees: AVL trees, red-black trees. Multiway search trees:

More information

Eastern Washington University Department of Computer Science. Questionnaire for Prospective Masters in Computer Science Students

Eastern Washington University Department of Computer Science. Questionnaire for Prospective Masters in Computer Science Students Eastern Washington University Department of Computer Science Questionnaire for Prospective Masters in Computer Science Students I. Personal Information Name: Last First M.I. Mailing Address: Permanent

More information

Parallel Databases. Parallel Architectures. Parallelism Terminology 1/4/2015. Increase performance by performing operations in parallel

Parallel Databases. Parallel Architectures. Parallelism Terminology 1/4/2015. Increase performance by performing operations in parallel Parallel Databases Increase performance by performing operations in parallel Parallel Architectures Shared memory Shared disk Shared nothing closely coupled loosely coupled Parallelism Terminology Speedup:

More information

Practical Database Design and Tuning

Practical Database Design and Tuning Practical Database Design and Tuning 1. Physical Database Design in Relational Databases Factors that Influence Physical Database Design: A. Analyzing the database queries and transactions For each query,

More information

1) The postfix expression for the infix expression A+B*(C+D)/F+D*E is ABCD+*F/DE*++

1) The postfix expression for the infix expression A+B*(C+D)/F+D*E is ABCD+*F/DE*++ Answer the following 1) The postfix expression for the infix expression A+B*(C+D)/F+D*E is ABCD+*F/DE*++ 2) Which data structure is needed to convert infix notations to postfix notations? Stack 3) The

More information

Quiz! Database Indexes. Index. Quiz! Disc and main memory. Quiz! How costly is this operation (naive solution)?

Quiz! Database Indexes. Index. Quiz! Disc and main memory. Quiz! How costly is this operation (naive solution)? Database Indexes How costly is this operation (naive solution)? course per weekday hour room TDA356 2 VR Monday 13:15 TDA356 2 VR Thursday 08:00 TDA356 4 HB1 Tuesday 08:00 TDA356 4 HB1 Friday 13:15 TIN090

More information

The Database is Slow

The Database is Slow The Database is Slow SQL Server Performance Tuning Starter Kit Calgary PASS Chapter, 19 August 2015 Randolph West, Born SQL Email: r@ndolph.ca Twitter: @rabryst Basic Internals Data File Transaction Log

More information

C H A P T E R 1 Introducing Data Relationships, Techniques for Data Manipulation, and Access Methods

C H A P T E R 1 Introducing Data Relationships, Techniques for Data Manipulation, and Access Methods C H A P T E R 1 Introducing Data Relationships, Techniques for Data Manipulation, and Access Methods Overview 1 Determining Data Relationships 1 Understanding the Methods for Combining SAS Data Sets 3

More information

2) What is the structure of an organization? Explain how IT support at different organizational levels.

2) What is the structure of an organization? Explain how IT support at different organizational levels. (PGDIT 01) Paper - I : BASICS OF INFORMATION TECHNOLOGY 1) What is an information technology? Why you need to know about IT. 2) What is the structure of an organization? Explain how IT support at different

More information