Binary Search Trees, etc.

Transcription

1 Chapter 11 Binary Search Trees, etc. The Dictionary data type is a collection of data, each of which has a key component. If we use an hashing table to represent a dictionary, it only takes, on average, Θ(1) to carry out such basic operations as search, insertion, and deletion. However, the worst time is O(n). It also can not provide good average time to carry out other operations such as output in ascending order of keys, find the k th element in ascending order, etc.. All those basic operations can be done in Θ(n), using a balanced search tree, a sorted output can be done in Θ(n), and the other operations can be done in Θ(log n). We will begin with binary search tree, which is not always balanced. 1

2 Binary Search Trees A binary search tree is a binary tree that may be empty. A nonempty binary tree has the following properties: 1) Every element has a key component, whose values are distinct. 2) The keys in the left subtree of the root are smaller than the key in the root. 3) The keys in the right subtree of the root are larger than that in the root. 4) Both the left and right subtrees are also binary search trees. An indexed BST is derived from a BST by adding a field, LeftSize, to each tree node, whose value is the number of nodes in its leftsubtree plus 1. 2

3 The BSTree class Since the number of nodes, as well as its shape, will change frequently, it is appropriate to use a linked structure to represent a binary search tree. Moreover, it will be convienient to derive it from the class of binary trees. For example, we can call the InOutput, operation defined for binary trees to output all the elements in an ascending order. class BSTree : public BinaryTree<E> { public: bool Search(const K& k, E& e) const; BSTree<E,K>& Insert(const E& e); BSTree<E,K>& Delete(const K& k, E& e); void Ascend() {InOutput();} }; 3

4 The search operation When we search for x, we begin at the root. If it is NULL, then x can t be found. Otherwise, we compare x with the key of the root. If they are equal, the search is a success. If it is larger that the key of the root, it must be in the right subtree, if it is inside the tree at all; so, we recursively search for x in the right subtree. Otherwise, we search the left one. bool BSTree<E,K>::Search (const K& k, E &e) const{ BinaryTreeNode<E> *p = root; while (p) // examine p->data if (k < p->data) p = p->leftchild; else if (k > p->data) p = p->rightchild; else { e = p->data; return true;} return false; } 4

5 Obviously, the search can be done in O(h), where h is the height of the tree. With an indexed BST, we can also search for the k th element, also in O(h). For example, to look for the third element in the following indexed BST, we check the LeftSize filed of the root, since it is 4, the element must be in the left subtree. We then check this field again for the root of its left subtree, it is a 2. hence, the element we are looking for is the smallest one of its right subtree. We find out that the LeftSize of its root is 1, i.e., so we can conclude that this node the root itself is the element we are looking for. 5

6 The insertion operation To insert an element with key k inside a BST, we first verify that this key does not occur anywhere inside the tree. If the search succeeds, the insertion will not be done; otherwise, the element will be inserted where the search reports a failure. The following are two examples: If we insert into an indexed BST, we also have to adjust the LeftSize field of all the elements from the insertion point back to the root. In both cases, it takes Θ(log n). 6

7 The code BSTree<E,K>& BSTree<E,K>::Insert(const E& e){ BinaryTreeNode<E> *p = root,*pp = 0; while (p){ pp = p; if (e < p->data) p = p->leftchild; else if (e > p->data) p = p->rightchild; else throw BadInput(); // duplicate } BinaryTreeNode<E> *r=new BinaryTreeNode<E> (e); if (root) { if (e < pp->data) pp->leftchild = r; else pp->rightchild = r;} else root = r; return *this; } 7

8 The deletion operation If the node to be deleted is a leaf, we simply delete it. It is also easy to do when it has just one child. When it has two children, it becomes a bit tricky, since we have only one location for two nodes to be hooked on. 8

9 BSTree<E,K>& BSTree<E,K>::Delete (const K& k, E& e){ BinaryTreeNode<E> *p = root, *pp = 0; while (p && p->data!= k){ pp = p; if (k < p->data) p = p->leftchild; else p = p->rightchild; } if (!p) throw BadInput(); e = p->data; if(p->leftchild && p->rightchild) { BinaryTreeNode<E> *s = p->leftchild,*ps = p; while (s->rightchild){ ps = s; s = s->rightchild;} p->data = s->data; p = s; pp = ps; } BinaryTreeNode<E> *c; if (p->leftchild) c = p->leftchild; else c = p->rightchild; if (p == root) root = c; else { if (p == pp->leftchild) pp->leftchild = c; else pp->rightchild = c;} delete p; return *this; } 9

10 BST could be high The height of a binary search tree can be quite high. For example, if we use the original insertion function to add in element 1, 2,...,n, in an empty BST, then the resulted BST is actually a linear list. As a result, a basic operation can take as must as O(n). But, if insertion and deletions are made at random, then the height of such a BST will be O(log n). 10

11 A little assignment Homework Generate a random permutation of the integers 1 through n, n [100, 500, 1000, 10000, 20000, 50000]. Insert them into an initially empty binary search tree according to the random order. Measure the height of the resulting search tree. Repeat this experiment for several random permutations and then calculate the average of the measured heights. Compare the just obtained average with the theoretical figure of 2 log 2 (n +1). 11

12 AVL trees We want to define the BST operations in such a way that the trees stay balanced all the times, while preserving the binary search tree properties. The essential algorithms for insertion and deletion will be exactly the same as for BST, except that as they might destroy the property of being balanced, we have to restore the balance of the tree, if needed. Such a tree is called a balanced tree, it is guaranteed that all the basic operations applied to a balanced tree will be done in Θ(log n). The class of AVL trees is one of the more popular balanced trees. An nonempty binary tree with T L and T R as its left and right subtrees is an AVL tree iff both T L and T R are AVL trees, and h L h R 1. 12

13 AVL search trees An AVL search tree is a binary search tree, which is also an AVL tree. It has the following properties. 1. The height of an AVL tree with n nodes is O(log n). 2. For every n 0, there exists an AVL tree with n nodes. 3. We can search an n element AVL search tree in O(log n). 4. A new element can be added into an n element AVL tree so that the resulted (n + 1) element tree is also an AVL tree. 5. An element can be deleted from an n element AVL search tree, and the resulted (n 1) element tree is also an AVL tree, and the deletion can be done in O(log n). 13

14 Fibonacci tree We calculate the least number of nodes in a balanced binary tree with depth h first: Let S(h) be this number. Obviously, S(0) = 1 and S(1) = 2. In general, we can construct a balanced tree with depth h by merging two subtrees of heights h 1 and h 2, each of which has the least number of nodes. We call such trees Fibonacci trees for the obvious reason. By an inductive argument, we have that S(h) = S(h 1) + S(h 2) + 1 fib(h), which leads to that S(h) > 3 2h 1. Thus, the least number of nodes in such a tree, n 0, satisfy that n 0 > 3 2h 1. 14

15 Height of an AVL tree Therefore, given any AVL tree with n nodes, we have that n n 0 > 3 2 h 1 Further calculation leads to the following result, h 1.44 log(n +2) A simpler, but unproved, result is that h log(n +1)

16 Represent an AVL tree An AVL tree is usually represented using the linked representation for binary trees. To provide information to restore the balance, a balance factor is added to each node. For any node, this factor is defined to be the difference between the heights of its left and right subtrees. It is easy to see that any search takes O(log n). 16

17 Restore the Balance When a tree becomes unbalanced, as a result of either insertion or deletion, we can restore its balance by rotating the tree at and/or below the node where the imbalance is detected. Case 1: Assume that the left subtree of k 2 has height larger than that of its right subtree, caused by adding a node into subtree X or deleting a node from subtree Z. We apply a right rotation, which will restore the balance of the tree rooted at k 1. Notice that the above rotation is symmetric. 17

18 An example Below shows how to restore the balance of a tree by applying a single rotation. Notice that here we just added a node with value into the left subtree rooted at 7, labeled as k 1 which itself is the left subtree of the tree rooted at 8, labeled as k 2. 18

19 Restore the Balance Case 2: Suppose that the imbalance is caused by the right subtree rooted at k 1 being too high, whose balance can t be restored by a single rotation. We must apply two successive single rotations, as follows. When it is caused by the left subtree being too high, we have the following similar double rotation. 19

20 An example Below shows how to restore the balance of a tree by applying a double rotation. Notice that here we just added a node with value 14, labeled as k 2 into the left subtree rooted at 15, labeled as k 1, which itself is the right subtree of the tree rooted at 7, labeled as k 3. 20

21 Histogramming We start with a collection of n keys and must output a list of distinct keys and their frequencies. Assume that n =10, and keys = [2, 4, 2, 2, 3, 4, 2, 6, 4, 2]. When the range of those keys is quite small, this problem can be solved, in Θ(n), by using an array h[0..r], where r is the maximum key value, and letting h[i] store the frequency of i. 21

22 When the key value are not integers, the above procedure can t be used. Instead, we can sort the n keys, in Θ(n log n), then scan the sorted list from left to right to find the frequencies of respective keys. This solution can be further improved, when m, the number of distinct keys, is quite small. We can use a (balanced) BST tree, each of whose nodes contains two fields, key and frequency. We insert all the n keys into such a tree, when a key is already there, we simply increment its frequency by 1. This procedure takes an expected complexity of Θ(n log m). When a balanced tree is used, this complexity is guaranteed, even in the worst case. 22

23 m way search trees An m way search tree, when it is not empty, satisfies the following conditions: 1) each internal node has up to m children and contains between 1 and m 1 elements. 2) Every node with p elements has exactly p + 1 children. 3) For any node with p elements. Let k 1 <,...,< k p be the keys of those elements, and let c 0,...,c p be the p + 1 children. Then, the keys in all the elements in the subtree c 0 are smaller than k 1, the keys in all the elements in the subtree c p are bigger than k p, and the keys in all the elements in subtree c i are between k i and k i+1, 1 i p 1. 23

24 It is easy to do a search in such a tree. To insert an element, we search for it first. The place where the search terminates indicates an insertion point. But, if there are no space left, we have to add an node to the tree to contain this element. It is a bit trickier to do a deletion. An m way search tree of height h may have as few as h elements, and as many as m h 1. As a result, the height of such a tree ranges between log m (n+1) and n. For example, a 200- way tree of height 5 can hold up to elements. When the search tree is located on a disk, the disk access time is in proportional to O(h), where h is the height of the tree. Thus, we have to cut the height as much as possible. 24

25 B tree A B tree of order m is an m way search tree. When it is not empty, it satisfies the following: 1) The root has at least two children. 2) All internal nodes other than the root have at least m 2 children. 3) All the external nodes are at the same level. 25

26 2-3 trees For a B tree of order 3, as it is a 3-way search tree, each of its internal node can have no more than 3 children. On the other hand, it is also required that such a node should have at least 2 children. Thus, each internal node will have either 2 or 3 children. Thus, it is also called a 2-3 tree. 26

27 Height of a B tree of order m Let N be the total number of keys. Since such a tree is an m way search tree, we immediately have that h log m (N +1). On the other hand, let d be m 2. As the root has at least two children, each of the other internal node has at least d children, we have that the least number of nodes at each level is 1, 2, 2d,...,2d h 1. Hence, the total number of nodes is at least dh 1 d 1. Further calculation shows that (d 1)(n 1) h log d ( 2 +1). We have that n N 1 d 1 +1, thus, h log d ( N +1 ). 2 27

28 Operations of B-tree Search is done pretty straightforward. Basically, to look for a node, we start with the root, following the lead given by the keys, we eventually either find it, or report a failure. When inserting an element, because of the number restriction, i.e., each node contains at most m 1 keys in an m ary B tree, we might have to split a node along the way, which may even lead to the growth of a tree. When deleting a node, there are two cases, w.r.t. the nature of the node to be deleted. 1) It is deleted from a leaf. 2) It is to be deleted from a non-leaf. In case 2) we can replace this node with either the largest element from its left neighboring subtree, or the smallest from its right subtree, then to delete the substituting element. Hence, case 2 can always be converted to case 1. 28

29 In case 1, if the element is to be deleted from a node that has at least one more than the minimum required number, i.e., m 2 1 for an m ary tree, we simply delete it and write back the modified node. Otherwise, we will try to replace it with an element from either its left or right sibling. When both siblings don t have that extra element, we have to merge those two nodes with the element between them in the parent node. This might have a chain reaction, ending up reducing the height of the whole tree. 29

30 The ISAM method AVL trees and other balanced trees will provide good performance when the dictionary is small enough to fit into the internal memory. When we have to deal with larger quantity of data, we can get improved performance by using search trees of higher degree. Let s look at ISAM (indexed sequential access method,) which is a popular access method for external memory, first. In ISAM, the available disk space will be cut into blocks, which is the smallest unit of disk space that will be input or output, at the cost of a single seek operation. The dictionary elements are packed into blocks in ascending order. 30

31 For sequential access, the blocks are input in order, and the elements will be retrieved in ascending order. If each block contains m elements, then the number of disk access time is m 1. On the other hand, to support random access, an index is maintained, which keeps the largest key in each block. Since, the index holds as many keys as the number of blocks, and since a block usually holds many elements, this index can usually be kept in the internal memory. To search for a record, the index is searched first for the block that contains this record. Then, that block is retrieved, and searched internally for that record. Thus, a single data access is enough. 31

32 This technique can be generalized, when data have to be packed into several disks, whence each disk will keep an index, and there will be an overall index kept in the internal memory. When search for something in this case, the overall index will be searched first to determine the single disk that contains the record. Then, the block index for that disk will be retrieved in a disk access, and finally, the relevant block will be brought in to do an internal search. So, in total, just two disk accesses are needed. In the next slide, we will see a more specific example, using B tree to maintain those indexes. 32

33 An Example Assume that we have a 128MB hard disk divided into 2 16 pages of 2048 bytes each. We use 4 bytes to represent the addresses of those pages. We also assume that the whole disk is used to store just one file of N records, each of which needs 512 bytes. Thus, each page can store 4 such records. Further assume that the key for each record is 4 bytes long. Thus, the key and the disk address of each record will need 8 bytes, so that each page can hold 256 of them. If we keep these index information separate from the records, we need 256 N pages to store them. We also need N 4 pages to keep the records. Also, the total number of pages must be less than This leads to the fact that N 258, 111. Assume that the equality holds. 33

34 We will use a B-tree to hold the indices. As each page can hold 256 indices, we decide to use a tree of order 256. The formula tells us that the height of such a tree will be less than 3, 2.43 to be exact. Therefore, to get access to any record in the disk, it takes at most four accesses to find the index and one more to get the record itself for a total of 5 disk accesses. Can we store the whole B-tree in RAM with 4 MB, thus, only one disk access is needed? Each node of the tree must store 255 indices of 8 bytes each and 256 pointers of 4 bytes each, thus a single node costs 3064 bytes, which leads to the fact that at most 1368 nodes will fit the RAM. However, as the root may contain just 1 index and each of the other node may contain just 128 indexes, thus the total indices contained by those nodes might be just 174,977, which is less than the total number of records. Thus, the whole tree cannot fit into the RAM. 34