Relational Database Systems 2 4. Trees & Advanced Indexes

Transcription

1 Relational Database Systems 2 4. Trees & Advanced Indexes Christoph Lofi Benjamin Köhncke Institut für Informationssysteme Technische Universität Braunschweig

2 2 Storage Buffer Management is very important Holds DB blocks in primary memory DB block are made up of several FS blocks Find good strategies to have requested DB blocks available when needed Each block holds some meta data and row data Indexes drastically speed up queries Less blocks need to be scanned Primary Index On primary key attribute, usually influences row storage order Secondary Index On any attribute, does not influence storage order Relational Database Systems 2 Christoph Lofi - Benjamin Köhncke Institut für Informationssysteme 2

3 4 Trees & Advanced Indexes 4.1 Introduction 4.2 Binary Search Trees 4.3 Self Balancing Binary Search Trees 4.4 B-Trees Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 3

4 4.1 Introduction Indexes need a suitable data structure For efficient index look-ups search keys need to be ordered Remember: All indexes should be stored in a separate database file, not together with data A suitable number of DB blocks (adjacent on disk) is reserved at index creation time If the space is not sufficient, another file is created and linked to the original index file Search Key 1 Block Address 1 Search Key 2 Block Address 2 Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 4

5 4.1 Introduction Search within an index Bisection search possible: log 2 n ; O(log n) But usually indexes span several DB blocks If index is in n blocks, O(n) blocks need to be read from disk Example: search for Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 5

6 4.1 Introduction Maintenance of index is also difficult Insert a new search key with value 5! In worst case, all cells need to be shifted and all blocks need to be accessed Similar problem occurs when deleting a value Often: do not shift values, but mark key as deleted Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 6

7 4.1 Introduction In this lecture, we discuss more efficient multilevel data structures B-trees Prevalent in database systems Better access performance Much better update performance To understand B-trees better, we start by examining binary search trees Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 7

8 4.2 Binary Trees Binary trees are Rooted and directed trees Each node has none, one or two children Each node (except root) has exactly one parent 0/1 Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 8

9 4.2 Binary Trees Some naming conventions Nodes without children are called leaf nodes The depth of node N is the path length from the root The tree height is the maximum node depth If there is a path from node N1 to node N2, N1 is an ancestor of N2 and N2 is a descendant of N1 The size of a node N is the number of all descendants of N including itself A subtree of a node N is formed of all descendant nodes including N and the respective links root subtree red red Leaf nodes tree height = 3 red node: size = 3 depth = 1 Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 9

10 4.2 Binary Trees Properties of binary trees Full binary tree (or proper) Each node has either zero or two children Perfect binary tree All leaf nodes have the same depth With height h, contains 2 h nodes Height-balanced binary tree Depth of all leaf nodes differ by at most 1 With height h, contains between 2 h-1 and 2 h nodes Degenerated binary tree Each node has either zero or one child Behaves like a linked list: search in O(n) Full and Balanced Full and Perfect Degenerated Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 10

11 4.2 Binary Search Trees Binary search trees are binary trees with Each node has a unique value assigned There is a total order on all values Left subtree of a node contains only values less than node value Right subtree of a node contains only values larger than the node value Aiming for O(log n) search complexity Structurally resembles bisection search / Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 11

12 4.2 Binary Search Trees Constructing and inserting into binary search trees Values are inserted incrementally First value is root Additional values sink into tree Sink to left subtree if value smaller Sink to right subtree if value larger Attach to last node as left/right child, if subtree is empty Insert order of values does highly influence resulting and intermediate tree properties Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 12

13 4.2 Binary Search Trees Suppose insert order 57, 33, 42, 85, 17, 61, Insert Insert 33, 42 Degenerated Insert 85, 17 Full and Balanced Insert 61, 99 Perfect and Full Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 13

14 4.2 Binary Search Trees Suppose insert order 99, 85, 61, 57, 42, 33, Insert complexity is thus O(n) worst case O(log n) average case Degenerated Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 14

15 4.2 Binary Search Tree Search for a Key Start with root Recursive Procedure If node value = v Return node If node is leaf Value not found if v < node value Descend to left subtree Else Descend to right subtree Complexity: Average case: O(log n) Worst case: O(n) degenerated tree Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 15

16 4.2 Binary Search Tree Tree Traversal Accesses all nodes of the tree Pre-Order Visit node Traverse left subtree Traverse right subtree In-Order (sorted access) Traverse left subtree Visit node Traverse right subtree Post-Order Traverse left subtree Traverse right subtree Visit node Pre-Order: In-Order: Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 16

17 4.2 Binary Search Tree Deleting nodes has complexity O(n) worst case, O(log n) average case Locate the node to delete by tree search If node is leaf, just delete it If node has one child, delete node and attach child to parent If node has two children Replace either by a) in-order successor (the left-most child of the right subtree) b) in-order predecessor (the right-most child of the left subtree) Example: delete search key with value 57 a) b) Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 17

18 4.2 Binary Search Trees Summary Very simple, dynamic data structure Efficient on average O(log n) for all operations Can be very inefficient for degenerated cases O(n) for all operations 0/1 Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 18

19 4.3 Self-Balancing Binary Search Trees Observation: Binary Search Trees are very efficient when perfect or balanced Idea: Continuously optimize tree structure to keep tree balanced Popular Implementations AVL-Tree (classic example) Red-Black-Tree Splay-Tree Scapegoat-Tree Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 19

20 4.3 Self-Balancing Binary Search Trees Basic Concepts for Deletion: Global Rebuild (Lazy Deletion) Start with balanced tree Don t delete a node, just mark it as deleted Search algorithm scans deleted nodes, but does not return them If Rebuild Condition is met, rebuild the whole tree without the deleted nodes Rebuild as soon as half of the nodes are marked as deleted Complete rebuild can be performed in O(n) Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 20

21 4.3 Self-Balancing Binary Search Trees Global Rebuild (cont.) Search Efficiency n number of unmarked nodes Tree is balanced, contains max 2n nodes overall Number of accesses during search usually just increases by 1 O(log n) Delete Efficiency Global rebuild is in O(n) But only necessary after n deletions Amortized additional costs per deletion is O(1) Overall complexity Average: O(log n) Worst Case: O(n), if actual rebuild is performed Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 21

22 4.3 Self-Balancing Binary Search Trees Global Rebuild (cont.) Direct Deletion with Rebuild Similar complexity as with lazy deletion Increased per delete effort Reduced per search effort until rebuild Delete nodes as in normal binary trees Increment deletion counter c d Rebuild tree as soon as c d = n, reset c d Delete 57,33,42,61 Rebuild Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 22

23 4.3 Self-Balancing Binary Search Trees Basic Concepts for Insertion and Deletion: Local Balancing (Subtree Balancing) Start with balanced tree Insert/delete nodes normally If a subtree becomes too unbalanced, locally balance subtree to regain global balance Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 23

24 4.3 Self-Balancing Binary Search Trees To detect unbalanced subtrees, each node v needs to know the size v and the height h(v) of it s subtree Unbalanced Condition: (Height Balancing) Subtree is too unbalanced when h(left(v)) - h(right(v)) > α α is a constant which can be adjusted (for AVL, α=1) Alternative Unbalanced Condition: Subtree is too unbalanced when h(v) > α * log 2 v α is a constant which can be adjusted Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 24

25 4.3 Self-Balancing Binary Search Trees Local Balancing (cont.) After inserting a node, walk back the tree and update stored subtree statistics h(v) and v. If node v is too imbalanced, balance subtree of v 57 2, , 6 Height Imbalanced for α=1 2-0 = 2 > , , , , , , 1 key 17 1, , 1 h(v) v 5 0, 1 Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 25

26 4.3 Self-Balancing Binary Search Trees Local Balancing can be archived by Rebuilding the subtree O( v ) = O(n) the worst case However, O(log n) in average This operation is expensive. But in the context of DBMS, it may pay off as it can also consolidate and optimize physical storage locations Especially suited for disk based trees Rotating Only pointers are moved very efficient O(1) Does not change physical storage of nodes Especially suited for main-memory-based trees Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 26

27 4.3 Self-Balancing Binary Search Trees Local Balancing Rotating Simple Rotation (left, right) Pivot y right x x 3 1 y 1 2 left 2 3 Double Rotation (left-left, right-right, Rollercoaster) z right-right y right-right x y 4 x z 1 y x z 1 2 left-left left-left 3 4 Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 27

28 4.3 Self-Balancing Binary Search Trees Local Balancing Rotating Double Rotation (left-right, Zig-Zag) z left z right x y 4 x 4 y z 1 x y Double Rotation (right-left, Zig-Zag) Analogous to left-right Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 28

29 Self-Balancing Binary Search Trees The presented concepts can be combined in different ways to implement self-balancing trees AVL-Tree (classic example) Red-Black-Tree Splay-Tree Scapegoat-Tree Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 29

30 Self-Balancing Binary Search Trees Implementation: AVL-Trees Invented 1962 by Adelson-Velsky and Landis Uses Local Rebalancing with Rotations for Insertion and Deletion Unbalanced criterion: h(left(v)) - h(right(v)) > 1 Height difference of left and right subtree of v is 2 or more Height information is stored explicitly within nodes Update backtracking after each insert and delete Storage overhead of O(n) Guaranteed maximum height of 1.44 log 2 n Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 30

31 Self-Balancing Binary Search Trees Implementation: Scapegoat-Trees Invented 1993 by Galperin and Rivest Uses Global Rebuilding for Deletions Local Balancing with Rotations for Insertions Unbalanced criterion: h(v) > log 1/α v + 1; 0.5 α 1 Node statistics (height, size) determined dynamically during backtracking Only global statistics are stored Storage overhead of O(1) Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 31

32 4.4 Problems with Binary Search Trees Are binary trees really suitable for disk based databases? Yes and No Binary Trees are great data-structures for usage in internal memory But they have a very bad performance when stored on external storage (i.e. hard disks) 0/1 & = Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 32

33 4.4 Problems with Binary Search Trees Binary tree nodes have to be stored within hard disk blocks in linear fashion When tree is large, nodes are scattered among the blocks In worst case, a new block must be read from disk for every node accessed during search or traversal Every linearization scheme for binary trees has that problem Reading a block from disk is very expensive Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 33

34 4.4 Problems with Binary Search Trees Sample linearization Search for 42 In worst case needs to fetch 3 blocks from disk for just 4 nodes Problem is even worse for full tree traversal Tree: Disk/DB Blocks: Block 1 Block 2 Block 3 Block 4 Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 34

35 4.4 B-Trees B-Trees adapt concepts and techniques learned for binary trees and optimize them for harddisc storage Basic Ideas: Searching within a DB/disk block is very efficient Take advantage of static nature within a block Search can be performed in memory with bisection search Treat entire blocks as tree nodes Reading blocks from the disk is expensive Reduce block reads Most data resides in the leaf nodes Thus minimize the height of the tree Dramatically increase fan-out factor Tree becomes bushy Smaller serach path length Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 35

36 4.4 Block Search Trees First Improvement: Block Search Tree Nodes are complete DB blocks Each node can store up to q pointers p i and q-1 unique and ordered key entries k i : <p 1, k 1,, k q-1, p q > k i < k i+1 Pointers p i link to subtrees (or are empty). All keys in subtree of p i are less than k i and greater as k i-1 Node Pointers Key Value Node EN Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 36

37 4.4 Block Search Trees Locate a key k Recursive Procedure: Start with root node Use bisection search within the current node If key found Return it If key not found If there is a p i with k i-1 < k < k i» Follow p i and repeat algorithm with link node Else» Key not in tree Example: Locate EN Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 37

38 4.4 Block Search Trees Insert a key k Recursive Procedure: Start with root node Use bisection search within the current node If key found Key cannot inserted twice, abort If key not found If there is a p i with k i-1 < k < k i» Follow p i and repeat algorithm with link node Else» If there is space left in the node Insert key and restore sort order» Else Create new, empty node Insert k into new node Link new node to p i in current node such that with k i-1 < k < k i EN Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 38

39 4.4 Block Search Trees Insert a key : EN Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 39

40 4.4 Block Search Trees Delete a key k Start with root node Locate k If k is in leaf node, delete k from node and restore order If leaf node is now empty, delete the node If k is in internal node If no or only one directly adjacent pointer of k are used» Delete k and restore order If k is a separator between two used pointers,» If space in both subnodes is sufficient Union both nodes into one Delete k and restore order» Else Replace k with new separator key Either largest key in left node or smallest key in right node Any completely empty node is deleted as in binary search trees EN Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 40

41 4.4 Block Search Trees Delete a key : EN Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 41

42 4.4 Block Search Trees Delete a key : EN Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 42

43 4.4 Block Search Trees Delete a key : EN Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 43

44 4.4 Block Search Trees Block Search Trees have similar properties to Binary Search Trees Can be perfect, balanced or degenerated Assume height h=3; fan-out-factor q=2048; and total number of keys n Block Search Tree One node can store up to 2047 keys and 2048 links Perfect : n = 8581M Balanced : 4M n 8581M Degenerated : n = 6141 Binary Search Tree (Block tree with q=2) One node can store 1 key and up to 2 links Perfect : n = 7 Balanced : 3 < n 7 Degenerated : n = 3 Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 44

45 4.4 Block Search Trees Assume n = 1,000,000,000; fan-out-factor q=2048; and height h Block Search Tree Balanced : h = 3 Degenerated : h = 488,520 Binary Search Tree Balanced : h = 30 Degenerated : h = 1,000,000,000 During search, there is one disk access per tree height in worst case In this example, block search trees are already 10 times more efficient when balanced, 2000 times when unbalanced Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 45

46 4.4 Block Search Trees Summary BST Data structure optimized for disk storage Is very efficient in average case O(log n) for all operations Even better Average node-accesses to locate a key is log fan-out n» Fan-out usually in the order of several thousands» Binary tree averages only to log 2 n Accessing a node is expensive on disks, huge improvement Can be very inefficient for degenerated cases O(n) for all operations Better than binary trees, but still bad Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 46

47 4.4 B-Trees B-Tree are specialized Block Search Trees for disc-based Indexing Invented by Rudolf Bayer in 1971 Keys may be non-unique Tree is self-balancing No degenerated cases anymore EN Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 47

48 4.4 B-Trees Basic structure of a B-tree node Nodes contain key values and respective data (block) pointers to the actual data records Additionally, there are node pointers for the left, resp. right interval around a key value Key Value Data Pointer Tree Node Node Pointers Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 48

49 1, Adams, $ 887,00 2, Bertram, $19,99 3, Behaim, $ 167,00 4, Cesar, $ 1866,00 5, Miller, $179,99 6, Naders, $ 682,56 7, Ruth, $ 8642,78 8, Smith, $675,99 9, Tarrens, $ 99, B-Trees B-Trees as Primary Index EN Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 49

50 4.4 B-Trees All base operations similar to Block Search Tree with small changes Guaranteed fill degree Self-Balancing Each node contains between L(ower) and U(pper) links Usually 2* L = U Nodes are split during insertion as soon as they contain more than U-2 keys Nodes are unioned during deletion as soon as they contain less than L keys If complete node is created or deleted, use local rebalancing to re-balance tree Local rebuilding for disk-based storage, rotations for memory based storage EN Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 50

51 4.4 B-Trees All insertions start at the leaf nodes Search the tree to find leaf node where new element should be added If the leaf node contains fewer than the maximum legal number of elements (if leaf node < U) Insert the new element in the node and restore order Otherwise the leaf node is split into two nodes (node split) The median is chosen from among the leaf's elements and the new element Values less than the median are put in the new left node and values greater than the median are put in the new right node, with the median acting as a separation value That separation value is added to the node's parent, which may also cause it to be split If the splitting goes all the way up to the root, it creates a new root with a single separator value and two children Remember: the lower bound on the size of internal nodes does not apply to the root Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 51

52 4.4 B-Trees Deleting elements is problematic, because node sizes can decrease under the minimum number of elements (i.e. new node size < L) An element to be deleted in an internal node may be a separator for its child nodes Deletion from a leaf node Search for the value to delete If the value is in a leaf node, it can simply be deleted from the node Test if node has too few elements; in that case the tree has to be rebalanced Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 52

53 4.4 B-Trees Rebalancing after deletion If some leaf node is under the minimum size, some elements must be redistributed from its siblings to bring all children nodes again up to the minimum (stealing) If all siblings have only minimum size, the parent node is affected and has to hand over an element If the parent then falls under the minimum size, the redistribution must be applied iteratively up the tree Since the minimum element count does not apply to the root, making the root the only deficient node is not a problem Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 53

54 4.4 B-Trees The rebalancing strategy is to find a sibling of the deficient node which has more than the minimum number of elements (for stealing) Choose a new separator Move it to the parent node and redistribute the values in both original nodes to the new left and right children If the sibling node to the right of the deficient node has only the minimum number of elements, examine the sibling node to the left Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 54

55 4.4 B-Trees If both siblings have only the minimum number of elements: create a new node with all the elements from the deficient node & all the elements from one of its siblings & the separator in the parent between the two combined sibling nodes Remove the separator from the parent, and replace the two children it separated with the combined node. If that brings the number of elements in the parent under the minimum, repeat these steps with that deficient node, unless it is the root, since the root may be deficient Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 55

56 4.4 B-Trees Example: Steal Keys from Siblings (l=3) Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 56

57 4.4 B-Trees Example: Join Child Nodes (l=3) Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 57

58 4.4 B-Trees Each element in an internal node acts as a separation value for two subtrees. When such an element is deleted, there are two cases: Both of the two child nodes to the left and right of the deleted element have the minimum number of elements (L-1) and then can then be joined into a legal single node with (2L-2) elements One of the two child nodes contains more than the minimum number of elements. Then a new separator for those subtrees must be found. There are two possible choices: The largest element in the left subtree is the largest element which is still less than the separator The smallest element in the right subtree is the smallest element which is still greater than the separator Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 58

59 4.4 B-Trees Deletion from an internal node If the value is in an internal node, choose a new separator, remove it from the leaf node it is in, and replace the element to be deleted with the new separator This has deleted an element from child node so the deletion has been passed down the tree iteratively If the child is a leaf node the leaf node deletion procedure applies Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 59

60 4.4 B-Trees Example: Build a B-Tree Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 60

61 4.4 B-Trees Summary Very efficient data structure for disk storage O(log n) for all operations Even better Guaranteed maximum node-accesses to locate a key is Balanced binary tree guarantees only log 2 n) log fan out ( n ) Accessing a node is expensive on disks huge improvement No degenerated cases Self-Balancing rarely necessary as most updates affect just one node Wasted space decreased due to guaranteed minimal fill factor EN Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 61

62 4.4 B*Trees The B*Tree is a constrained B-Tree All non-root nodes need to be filled to 2/3 Implemented in various file systems HFS Raiser 4 Used to be quite popular, but lost its importance EN Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 62

63 4.4 B + Trees The B + Tree is an optimization of the B-Tree Improved traversal performance Increased search efficiency Increased memory efficiency B + Tree uses different nodes for leaf nodes and internal nodes Internal Nodes: Only unique keys and node links No data pointers! Leaf Nodes: Replicated keys with data pointer Data pointers only here Node Pointer Key Value Key Value Data Pointer EN Node Internal Node Leaf Node Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 63

64 4.4 B + Trees Internal Nodes are used for search guidance A block can contain more keys fan-out higher Leafs just contain data links All leafs are linked to each other in-order for increased traversal performance Internal Search Nodes Data Nodes EN Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 64

65 4.4 B + Trees Summary B + Tree is THE super index structure for disk-based databases Improved over B-Tree Improved traversal performance Increased search efficiency Increased memory efficiency EN Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 65

66 4.5 IMDB s Observation Loading data from the hard disk is a major bottleneck Available main memory still doubles every 18 month Moore s Law Idea Store all data in fast main memory! Solutions Use traditional DBMS with huge buffer pool (block cache) DBMS are usually optimized for sequential disk access Design special In-Memory Databases Systems Or MMDB (Main Memory Database) Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 66

67 4.5 IMDB s Why do we need in-memory databases? Embedded Systems Mobile Phones PDA s Sensors Diskless Computing Devices Ultra-High-Performance (Real Time) Scenarios Network Applications Telecommunication Applications High-Volume Trading Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 67

68 4.5 IMDB s Why should IMDB s be different? Traditional DBMS do also work in-memory but they waste potential Random access has nearly no penalty compared to sequential access Optimizing for linear storage and block read/write unnecessary Type Media Size Random Acc. Speed Transfer Speed Characteristics Price Price/GB Pri DDR3-Ram (Corsair 1600C7DHX) 2 GiB ms 8000 MB/sec Vol, Dyn, Ra, OL Sec Harddrive Magnetic (Seagate ST AS) 2000 GB < 8.5 ms 138 MB/sec Stat, RA, OL Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 68

69 4.5 IMDB s Storing a DB in main memory also has problems Main memory usually smaller and more expensive ACID support IMDBs support atomicity, consistency and isolation Problem: main memory is not persistent What happens in case of power failure? How to ensure the durability requirement of DBs? Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 69

70 4.5 IMDB s - Durability Snapshot Files / Checkpoint Images Record state of DB at given point in time Done periodically or in case of controlled shutdown Only partial durability Transaction Logging History of actions executed by the DBMS File of changes done in the database stored in stable storage If DB has not been shut down properly (respectively is in inconsistent state) the DBMS reviews logs for uncommitted transactions and rolls back changes Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 70

71 4.5 IMDB s - Durability Non-volatile random access memory (NVRAM) Static RAM backed up with battery power (battery RAM) Or electrically erasable programmable ROM (EEPROM) Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 71

72 4.5 IMDB s - Indexes IMDB Index Structures B-Trees are great, but they are shallow and bushy which is unnecessary in main memory Can save some performance there Hash Indexes are very suitable in main memory for unsorted data Especially bucket chained hashing is very efficient For sorted data: Use the T-Tree instead of B-Tree Specialized tree for main memory databases Blend between AVL-Tree and B-Tree Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 72

73 4.5 T-Tree T-Tree design considerations I/O access is cheap in main memory Expensive resources are computation time and memory space Properties T-Tree is a self-balancing binary tree (AVL algorithm) T-Tree nodes contain only links Each node links to m data records (d 1 d m ) Data entries are ordered, smallest left, biggest right All nodes contain a maximum of c max entries Each internal node contains c min to c max entries (usually c max -c min 2) Each node has a link to it s parent Each node has at most a left and a right subtree Left subtree contains only entries smaller than the minimal node entry Right subtree contains only entries bigger than maximal node entry d 1 p d 2 d m-1 d m l r Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 73

74 4.5 IMDB Indexes How do main memory index structures compare? Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 74

75 4.5 IMDB Indexes How do main memory index structures compare? Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 75

76 4.5 IMDB Indexes How do main memory index structures compare? Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 76

77 4.5 IMDB Indexes Why not always use Chained Bucked Hashing? No range queries Storage overhead Suboptimal if amount of data is unknown during initialization Why do T-Tree and AVL-Tree perform better than B- Tree and ordered array for search? Bisection search within a B-tree node/array needs to compute position of next comparison AVL and T-Tree do only need 2 comparisons in each node Why does T-Tree perform better than AVL for updates? Due to larger nodes, many updates do not require a rebalancing Why does ordered array suck for updates? Reordering of all elements necessary for each update Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 77

78 4.5 References and Timeline AVL-Tree G. Adelson-Velskii, E. M. Landis: An algorithm for the organization of information. Proceedings of the USSR Academy of Sciences 146: (Russian), English translation by M. J. Ricci in Soviet Math. Doklady, 3: , 1962 B-Trees R. Bayer, E. M. McCreight: Organization and Maintenance of Large Ordered Indexes. Acta Informatica 1, , 1972 T-Trees T. J. Lehman, M. J. Carey: A Study of Index Structures for Main Memory Database Management Systems, Int. Conf. On Very Large 12th Database, Kyoto, August 1986 Scapegoat Trees I. Galperin, R. L. Rivest: Scapegoat trees, ACM-SIAM Symposium on Discrete Algorithms, Austin, Texas, US, 1993 Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 78

79 4 Storage Tree data structures are good index structures O(log n ) performance in average But they may degenerate O(n) Balancing necesarry! Block structure of hard discs must be considered Block trees B-Tree Self-balancing block tree with fill-guarantees B+-Tree Special inner nodes without data pointers Leaf nodes optimized for linear traversal Relational Database Systems 2 Christoph Lofi - Benjamin Köhncke Institut für Informationssysteme 79

80 5 Outlook The Query Processor How do DBMS actually answer queries? Query Parsing/Translation Query Optimization Query Execution Implementation of Joins Datenbanksysteme 2 Christoph Lofi Institut für Informationssysteme TU Braunschweig 80