# Lecture 6. Randomized data structures Random number generation Skip lists: ideas and implementation Skip list time costs. Reading:

Save this PDF as:

Size: px
Start display at page:

Download "Lecture 6. Randomized data structures Random number generation Skip lists: ideas and implementation Skip list time costs. Reading:"

## Transcription

1 Lecture 6 Randomized data structures Random number generation Skip lists: ideas and implementation Skip list time costs Reading: Skip Lists: A Probabilistic Alternative to Balanced Trees (author William Pugh, available online); Weiss, Chapter 10 section 4 Page 1 of 43

2 Deterministic and probabilistic data structures Most typical data structures are deterministic data structures: starting with an empty structure and a sequence of insert, find, and delete operations with specific key values, you will always get exactly the same resulting structure In the last few years, there has been a lot of interest in probabilistic, or randomized, algorithms and data structures: in a probabilistic data structure, the same sequence of operations will usually not result in the same structure; it depends on the sequence of random bits generated during the operations It turns out that these probabilistic data structures can work about as well as the best deterministic ones, but can be much easier to implement Skip lists and randomized search trees are examples of randomized data structures We will look at skip lists in a moment But first, we will briefly discuss random number generation Page 2 of 43

3 Random number generation Random numbers are useful in many applications: cryptography, software testing, games, probabilistic data structures, etc A truly random number would be truly unpredictable However, ordinary programs are deterministic, and therefore their outputs are predictable, if you know the inputs, and the algorithm! So, you can t write an ordinary program to generate truly random numbers; you generate pseudorandom numbers (you could connect your computer to a nonalgorithmic device and collect truly random bits from it; see /dev/random on Unix systems, or various web services) A typical pseudorandom number generator (RNG) is initialized with a seed; if the seed is truly random, the RNG will have as much randomness as there is in the seed Though the generated numbers aren t truly random, they should appear quite random what does that mean? Page 3 of 43

4 Tests for randomness A good pseudorandom number generator (RNG) will generate a sequence of numbers that will pass various statistical tests of randomness the numbers will appear to be random, and can be used in many applications as if they were truly random Some good properties of a random sequence of numbers: Each bit in a number should have a 50% probability of changing (1->0 or 0->1) when going from one number to the next in the sequence The sequence of numbers should not repeat at all, or if the numbers do cycle, the cycle should be very (very!) long How to write a program that generates good pseudorandom numbers? One common approach is the linear congruential RNG Page 4 of 43

5 Linear congruential RNG s A typical pseudorandom number generator works like this: The RNG is initialized by setting the value of a seed The sequence of numbers generated is determined by the value of the seed; if you want the same sequence again, start with the same seed If the seed has 64 bits, and those bits are picked truly randomly (eg by flipping a fair coin) then the RNG sequence has only 64 bits of true randomness To generate the next pseudorandom number, the RNG updates the value of the seed, and returns it (or some part of it): seed = F(seed); return partof(seed); The function F can take various forms If it has the form F(x) { return (a * x + c) % m ; } for some constants a, c, m, then this is a linear congruential RNG: it computes a linear (or affine) function of the seed, with the result taken congruent with respect to (ie, modulo) m (Interesting to compare this to some hash functions used for strings) Page 5 of 43

6 Linear congruential generators, cont d How to choose the constants a, c, m for the generator? Not so easy to do! Many values for these constants produce RNG s whose sequences aren t very random at all The following values work well, and are used in some C/C++ standard library implementations: a = c = m = 1<<31 The generator in the javautilrandom class uses the following values This RNG s low-order bits are not very random; they have a short cycle So, computation is done with 64-bit longs, and the low order 16 bits are not exposed to the user: a = c = 11 m = 1<<48 Other fancier approaches are needed for highly-secure cryptography, etc Page 6 of 43

7 Pseudorandom numbers in C++ The rand() and srand(int) functions are defined in the C standard library, for use in C and C++ programs The rand() function returns pseudorandom numbers in the range [0,RAND_MAX] #include <cstdlib> to define the const int RAND_MAX srand(x)sets the seed for rand() s generator If rand() is called in your program before any call to srand(int), it first calls srand(1) Call srand(int)at any time to reset the seed call srand(1) to restart the default sequence #include <ctime>, and call srand(time(0)) to get a seed that depends on the current system clock time, but only to 1 second resolution Page 7 of 43

8 Toward skip lists Skip lists are a data structure that uses randomization in its construction One way to understand the idea of skip lists start with ordinary linked lists then think about how to make search in a linked list more like binary search in an array Page 8 of 43

9 Fast searching in lists As you know, binary search in a sorted array is very fast: O(log N) worst-case the direct-access property of arrays lets you compare to the middle element of the array in constant time; and then to the middle element of the appropriate remaining half of the array in constant time; etc each comparison eliminates (about) half of the remaining possibilities However, search in a linked list is slow, even if the list is sorted: O(N) average case the sequential-access property of linked lists forces a linear search How to make this faster? Page 9 of 43

10 Fast searching in lists, cont d Suppose every other list node, besides having a pointer to the next node in the list, also has a pointer two nodes ahead: then search can be about twice as fast as before: Start following the level 2 pointers, until it leads to a node with a key larger than the one you are looking for; then drop down to the level 1 pointers If every fourth node also has a pointer four nodes ahead, search can be 4 times as fast: Start following the level 3 pointers, until it leads to a node with a key larger than the one you are looking for; then drop down to the level 2 pointers, etc If you fill this out, so that for J = 0,,logN, 1/2 J of the nodes in the list have a pointer 2 J nodes ahead, you get a structure as shown on the next page with worst-case search cost O(log N) Page 10 of 43

11 Pointers for fast searching in a linked list Page 11 of 43

12 Fast lists, binary search and BST s The added pointers in that enhanced fast list permits binary search in a linked list Note that a well-balanced binary search tree has the same property: each comparison eliminates about half of the remaining possibilities In fact, searching in that example structure will visit nodes in the same sequence as in this binary search tree: Page 12 of 43

13 Skip lists The enhanced linked list structure shown permits fast O(logN) search, worst case However, to preserve that perfect nice structure as an invariant when doing an insert or delete operation would require O(N) steps worst-case, because every node may need its pointers changed Pugh [1990] showed that in practice you really don t need to preserve a perfectly regular structure when doing insert and delete operations: Instead, you can use a random number generator during insert operations, and create a randomized structure, and still almost certainly get very good performance The resulting structure is called a skip list Skip lists are relatively easy to implement, and have expected time cost O(logN) for search, insert, and delete They have worst case time cost O(N), but this worst case is extremely unlikely to occur if you use a good random number generator Page 13 of 43

14 Skip lists: some definitions In a skip list, nodes in the list contain different numbers of forward pointers The forward pointers in a node are stored in an array in that node, and they point to nodes farther down the skip list (or are null) The level of a node is defined to be the length of its array of forward pointers The skip list has associated with it a probability p, 0<p<1 This determines the probability of the level of a new node When creating a new node, flip a coin that has probability p of heads Count the number of heads before the first tails, plus one: that is the level of the new node Common choices for p are 1/2 and 1/4: With p = 1/2, the probability of a new node being level 1 is 1/2; of being level 2 is 1/4; of being level 3 is 1/8; etc With p = 1/4, the probability of a new node being level 1 is 3/4; of being level 2 is 3/16; of being level 3 is 3/64; etc Page 14 of 43

15 Skip list node level distribution: p = 1/2 Suppose you have a completely filled binary tree with 2 K - 1 nodes How many nodes are at the highest level (ie the root)? What fraction is that of the total? How many nodes are at the next highest level (ie children of the root)? What fraction is that of the total? How many nodes are at the lowest level (ie the leaves)? What fraction is that of the total? Using 1-based counting of levels going up from the leaves, the fraction of nodes at level L from the leaves is 2 K L 2K 1, which is approximately 1 2 L The idea behind a skip list with p = 1/2 is to approximate that distribution of nodes at different levels, in the hope of getting performance similar to a completely filled binary search tree Page 15 of 43

16 Skip list node level distribution: p = 1/4 Suppose you have a completely filled 4-ary tree with (4 K - 1)/3 nodes How many nodes are at the highest level (ie the root)? What fraction is that of the total? How many nodes are at the next highest level (ie children of the root)? What fraction is that of the total? How many nodes are at the lowest level (ie the leaves)? What fraction is that of the total? Using 1-based counting of levels up from the leaves, the fraction of nodes at level L from the leaves is 4 K L 4 K 1 3, which is approximately 3 4 L The idea behind a skip list with p = 1/4 is to approximate that distribution of nodes at different levels, in the hope of getting performance similar to a completely filled 4-ary search tree Page 16 of 43

17 Skip lists: some definitions, cont d Every node has at least one pointer (index 0 in that node s pointer array), and at most MAXLEVEL pointers MAXLEVEL should be chosen to be approximately log 1/p of the maximum number of items that will be stored in the skip list if many more than that many items are stored in the skip list, performance will deterioriate somewhat if many fewer than that many items are stored, you are wasting some space storing pointers that don t really help performance The index 0 forward pointer in a node points to the next node in the list (or null if none) The index j pointer points to the next node in the list of level greater than j (that is, the next node that contains a pointer with index j), or null if none A skip list has a header node that contains MAXLEVEL pointers, all initially null Page 17 of 43

18 Skip list: example Here is a skip list containing keys 3, 5, 6, 21, 22, 25, 26 with ordinary integer ordering Levels randomly generated for nodes containing those keys are 2,1,3,1,2,4,1 In this example, MAXLEVEL == 4 header: key Note that the sequence of keys in the skip list is determined by the value of the keys only; the levels of the nodes are determined by the random number generator only Page 18 of 43

19 Insert in skip lists When inserting a key in a skip list, a new node is created to hold the key The sequence of nodes in a skip list is always maintained in key-sorted order, so the postion of the new node in the list is determined by the key values However, the level of the new node is selected randomly and independently of the value of the key: the randomly selected level is a number in the range [1,MAXLEVEL] the probability that the level will be 1 is (1-p) the probability of a node that has a level i pointer NOT having a level i+1 pointer is (1 - p) unless i == MAXLEVEL (because no nodes have pointers at level greater than MAXLEVEL ) The level of a node is determined by a random number generator when the node is created, and, once determined, it is never changed (of course, the node may be deleted later) Page 19 of 43

20 Skip list implementation Skip list nodes need to hold a key, and an array or vector of forward pointers The keys need to be comparable to each other The length of the pointer vector is equal to the level of the node template <typename T> class SkipNode { friend class SkipList; // SkipList class can get access to // all SkipNode members private: T* key; // pointer to the key in this node std::vector<skipnode<t>*> forward; // vector of forward pointers }; SkipNode(T* key, int level) : key(key), forward(level,(skipnode<t>*)0) {} Page 20 of 43

21 The STL std::vector container std::vector is a very useful, high performance container class You can think of a vector as a safe, resizeable array Some vector features we will make use of: A constructor that takes two arguments (there are also other constructors): an unsigned int, saying how many elements the vector has a value that will be used to initialize those elements of the vector operator[] overloaded so a vector can be indexed like a native array member function at(unsigned int i) which acts like the indexing operator[], but which does runtime index bounds checking Page 21 of 43

22 Skip list implementation, cont d The skip list will need a vector of MAXLEVEL forward pointers This can be implemented using a dummy header SkipNode with null data template <typename T> class SkipList { private: SkipNode<T> header; const int MAXLEVEL; float prob; public: SkipList(int maxlevel, float p) : MAXLEVEL(maxlevel), header((t*)0, maxlevel) { prob = p < 10 && p > 00? p : 05; } }; void insert(t* key); // will define this later Page 22 of 43

23 Random level generation Use a random number generator to decide the level of a new node to be inserted in the skip list Flip a coin that has probability p of being heads: the number of heads in succession up to the first tails, plus one, will be the randomly selected level So, probability of a node being level 1 is 1-p, probability of being level 2 is p(1-p), probability of being level 3 is p 2 (1-p), etc (Need to make sure we don t exceed MAXLEVEL) private: /** Return a random double in range [0,1) */ double drand() { return rand() / (RAND_MAX + 10); } int randomlevel() { int level; // note the empty loop body! for(level = 1; drand() < prob && level < MAXLEVEL; level++) ; return level; } Page 23 of 43

24 Find in skip lists To find key k: 0 Start at list header, and consider it to be the current node x Find the highest index pointer in the header that is not null; let this index be i 1 While the pointer in node x indexed i is not null and points to a node containing a key smaller than k, set that node pointed to to be the current node x (ie follow the pointer) 2 If the pointer in node x indexed i points to a node containing a key equal to k, you have found the key k in the skip list and you are done 3 If the pointer in node x indexed i points to a node containing a key larger than k, decrement i (ie, keep x as the current node but drop down to the pointer with next smaller index) 4 If now i < 0 (ie, you have run out of pointers in the current node), the key k is not in the skip list and you are done Else, go to 1 Page 24 of 43

25 Insert in skip list Basic idea: Use the skip list search algorithm to find the place in the skip list to insert the new key (recall that keys are kept in sorted order in the list) While searching, use a local array of update pointers to keep track of previous nodes at each level, so their pointers can be easily updated when you splice in the new node Create a new node with a random level to hold the new key, and splice it in Page 25 of 43

26 Inserting in a skip list: example Starting with an empty skip list with MAXLEVEL 4, insert these keys, with these randomly generated levels 5, with level 1 26, with level 1 25, with level 4 6, with level 3 21, with level 1 3, with level 2 22, with level 2 Page 26 of 43

27 Insert 5 with level 1 header: key 5 Page 27 of 43

28 Insert 26 with level 1 header: key 5 26 Page 28 of 43

29 Insert 25 with level 4 header: key Page 29 of 43

30 Insert 6 with level 3 header: key Page 30 of 43

31 Insert 21 with level 1 header: key Page 31 of 43

32 Insert 3 with level 2 header: key Page 32 of 43

33 Inserting 22 with level 2 Let s look at the example of inserting this next key in more detail The insert algorithm maintains two local variables (besides the skip list header): x, a pointer which points to a node whose forward pointers point to nodes whose key we are currently comparing to the key we want to insert this lets us quickly compare keys, and follow forward pointers update, a vector of node pointers which point to nodes whose forward pointers may need to be updated to point to the newly inserted node, if the new node is inserted in the list just before the node x points to this lets us quickly update all the pointers necessary to splice in the new node 6 keys have been inserted, and see how key 22 is inserted in a new node with level 2 Page 33 of 43

34 Inserting 22 with level 2: frame 1 Follow top-level pointer: 25 > 22, so drop down and follow pointer: 6 < 22, so update header: key update: x: 0 Page 34 of 43

35 Inserting 22 with level 2: frame 2 Follow pointer: 25 > 22, so drop down twice and follow pointer: 21 < 22, so update header: key update: x: 0 Page 35 of 43

36 Inserting 22 with level 2: frame 3 Follow pointer: 25 > 22 and we are at lowest level, so we have found insert point header: key update: x: 0 Page 36 of 43

37 Inserting 22 with level 2: frame 4 The update array shows which nodes forward pointers are involved in the insert header: key update: x: 0 Page 37 of 43

38 Here is the skip list after all keys have been inserted: header: key Note that the ordering of the keys in the skip list is determined by the value of the keys only; the levels of the nodes are determined by the random number generator only Page 38 of 43

39 Insert in skip lists: code template <typename T> void SkipList<T>::insert(T* key) { std::vector<skipnode<t*>> update(maxlevel,0); // "update" vector SkipNode<T>* x = &header; // start at dummy header node for(int i=maxlevel-1; i>=0; i--) { // search for insert posn while( x->forwardat(i) && *(x->forwardat(i)key) < *key ) { x = x->forwardat(i); // follow pointer with index i } update[i] = x; // keep track of end of chain at index i } int newlevel = randomlevel(); // generate level of the new node x = new SkipNode<T>(key, newlevel); // create new node } // splice into list for (int i=0; i<newlevel; i++) { x->forward[i] = update[i]->forward[i]; // who x points to update[i]->forward[i] = x; // who points to x } Page 39 of 43

40 Delete in skip lists Basic idea, similar to insert: Use the skip list search algorithm to find the node to delete from the list The update array is important here too: it keeps track of nodes whose pointers may need to change to unsplice the to-be-deleted node Page 40 of 43

41 Properties of skip lists It is possible for a skip list to be badly unbalanced, so that searching in it is like searching in a linked list for example, this will happen if the levels of the nodes in the list are nonincreasing (or nondecreasing) as you move from the beginning of the list to the end So, it is possible that the time cost of operations in a skip list will be O(N) However, Pugh shows that the expected number of comparisons to do a find in a skip list with N nodes using probability p for generating levels is E comparisons log p 1 p N p (For p = 1/2, this implies that the average number of comparisons should be no more than about 2 log 2 N, which can be compared to about 1386 log 2 N for a randomized search tree) And so the expected time costs for insert and delete are also O(logN) Page 41 of 43

42 Properties of skip lists, cont d The expected time costs are like average time costs, averaged over many constructions of a skip list with the same N keys, but different randomly chosen node levels An interesting question is: For a single skip list with N keys, how likely is it that its height is much greater than log N? Pugh analyzed this question and, though he did not publish a formula to compute it, he shows graphs which indicate that the answer is: it is possible, but it can be considered extremely unlikely Overall, skip lists are comparable to randomized search trees in time costs of the basic operations memory cost implementation complexity Page 42 of 43

43 Next time Red-black trees Red-black tree properties Insert in red-black trees: rotations and recolorings Readings: Weiss, Ch 12 section 2 Page 43 of 43

### Binary Search Trees. Data in each node. Larger than the data in its left child Smaller than the data in its right child

Binary Search Trees Data in each node Larger than the data in its left child Smaller than the data in its right child FIGURE 11-6 Arbitrary binary tree FIGURE 11-7 Binary search tree Data Structures Using

### - Easy to insert & delete in O(1) time - Don t need to estimate total memory needed. - Hard to search in less than O(n) time

Skip Lists CMSC 420 Linked Lists Benefits & Drawbacks Benefits: - Easy to insert & delete in O(1) time - Don t need to estimate total memory needed Drawbacks: - Hard to search in less than O(n) time (binary

### Symbol Tables. Introduction

Symbol Tables Introduction A compiler needs to collect and use information about the names appearing in the source program. This information is entered into a data structure called a symbol table. The

### A Comparison of Dictionary Implementations

A Comparison of Dictionary Implementations Mark P Neyer April 10, 2009 1 Introduction A common problem in computer science is the representation of a mapping between two sets. A mapping f : A B is a function

### A binary search tree or BST is a binary tree that is either empty or in which the data element of each node has a key, and:

Binary Search Trees 1 The general binary tree shown in the previous chapter is not terribly useful in practice. The chief use of binary trees is for providing rapid access to data (indexing, if you will)

### CS104: Data Structures and Object-Oriented Design (Fall 2013) October 24, 2013: Priority Queues Scribes: CS 104 Teaching Team

CS104: Data Structures and Object-Oriented Design (Fall 2013) October 24, 2013: Priority Queues Scribes: CS 104 Teaching Team Lecture Summary In this lecture, we learned about the ADT Priority Queue. A

### Binary Heap Algorithms

CS Data Structures and Algorithms Lecture Slides Wednesday, April 5, 2009 Glenn G. Chappell Department of Computer Science University of Alaska Fairbanks CHAPPELLG@member.ams.org 2005 2009 Glenn G. Chappell

### Chapter Objectives. Chapter 9. Sequential Search. Search Algorithms. Search Algorithms. Binary Search

Chapter Objectives Chapter 9 Search Algorithms Data Structures Using C++ 1 Learn the various search algorithms Explore how to implement the sequential and binary search algorithms Discover how the sequential

### A binary search tree or BST is a binary tree that is either empty or in which the data element of each node has a key, and:

1 The general binary tree shown in the previous chapter is not terribly useful in practice. The chief use of binary trees is for providing rapid access to data (indexing, if you will) and the general binary

### CS 2112 Spring 2014. 0 Instructions. Assignment 3 Data Structures and Web Filtering. 0.1 Grading. 0.2 Partners. 0.3 Restrictions

CS 2112 Spring 2014 Assignment 3 Data Structures and Web Filtering Due: March 4, 2014 11:59 PM Implementing spam blacklists and web filters requires matching candidate domain names and URLs very rapidly

### recursion, O(n), linked lists 6/14

recursion, O(n), linked lists 6/14 recursion reducing the amount of data to process and processing a smaller amount of data example: process one item in a list, recursively process the rest of the list

### Sorting revisited. Build the binary search tree: O(n^2) Traverse the binary tree: O(n) Total: O(n^2) + O(n) = O(n^2)

Sorting revisited How did we use a binary search tree to sort an array of elements? Tree Sort Algorithm Given: An array of elements to sort 1. Build a binary search tree out of the elements 2. Traverse

### Merge Sort. 2004 Goodrich, Tamassia. Merge Sort 1

Merge Sort 7 2 9 4 2 4 7 9 7 2 2 7 9 4 4 9 7 7 2 2 9 9 4 4 Merge Sort 1 Divide-and-Conquer Divide-and conquer is a general algorithm design paradigm: Divide: divide the input data S in two disjoint subsets

### From Last Time: Remove (Delete) Operation

CSE 32 Lecture : More on Search Trees Today s Topics: Lazy Operations Run Time Analysis of Binary Search Tree Operations Balanced Search Trees AVL Trees and Rotations Covered in Chapter of the text From

### 1) The postfix expression for the infix expression A+B*(C+D)/F+D*E is ABCD+*F/DE*++

Answer the following 1) The postfix expression for the infix expression A+B*(C+D)/F+D*E is ABCD+*F/DE*++ 2) Which data structure is needed to convert infix notations to postfix notations? Stack 3) The

### Lecture Notes on Binary Search Trees

Lecture Notes on Binary Search Trees 15-122: Principles of Imperative Computation Frank Pfenning André Platzer Lecture 17 October 23, 2014 1 Introduction In this lecture, we will continue considering associative

### B-Trees. Algorithms and data structures for external memory as opposed to the main memory B-Trees. B -trees

B-Trees Algorithms and data structures for external memory as opposed to the main memory B-Trees Previous Lectures Height balanced binary search trees: AVL trees, red-black trees. Multiway search trees:

### Cpt S 223. School of EECS, WSU

Priority Queues (Heaps) 1 Motivation Queues are a standard mechanism for ordering tasks on a first-come, first-served basis However, some tasks may be more important or timely than others (higher priority)

### Heaps & Priority Queues in the C++ STL 2-3 Trees

Heaps & Priority Queues in the C++ STL 2-3 Trees CS 3 Data Structures and Algorithms Lecture Slides Friday, April 7, 2009 Glenn G. Chappell Department of Computer Science University of Alaska Fairbanks

### Questions 1 through 25 are worth 2 points each. Choose one best answer for each.

Questions 1 through 25 are worth 2 points each. Choose one best answer for each. 1. For the singly linked list implementation of the queue, where are the enqueues and dequeues performed? c a. Enqueue in

### Node-Based Structures Linked Lists: Implementation

Linked Lists: Implementation CS 311 Data Structures and Algorithms Lecture Slides Monday, March 30, 2009 Glenn G. Chappell Department of Computer Science University of Alaska Fairbanks CHAPPELLG@member.ams.org

### TIE-20106 1 TIE-20106 2

TIE-20106 1 1 List structures In this chapter, a few simple data structures (stack and queue) that can be implemented with arrays and/or lists are covered. Two more complex structures based on lists are

### Converting a Number from Decimal to Binary

Converting a Number from Decimal to Binary Convert nonnegative integer in decimal format (base 10) into equivalent binary number (base 2) Rightmost bit of x Remainder of x after division by two Recursive

### Binary Search Trees CMPSC 122

Binary Search Trees CMPSC 122 Note: This notes packet has significant overlap with the first set of trees notes I do in CMPSC 360, but goes into much greater depth on turning BSTs into pseudocode than

### root node level: internal node edge leaf node CS@VT Data Structures & Algorithms 2000-2009 McQuain

inary Trees 1 A binary tree is either empty, or it consists of a node called the root together with two binary trees called the left subtree and the right subtree of the root, which are disjoint from each

### Quiz 4 Solutions EECS 211: FUNDAMENTALS OF COMPUTER PROGRAMMING II. 1 Q u i z 4 S o l u t i o n s

Quiz 4 Solutions Q1: What value does function mystery return when called with a value of 4? int mystery ( int number ) { if ( number

### 1. The memory address of the first element of an array is called A. floor address B. foundation addressc. first address D.

1. The memory address of the first element of an array is called A. floor address B. foundation addressc. first address D. base address 2. The memory address of fifth element of an array can be calculated

### ECE 250 Data Structures and Algorithms MIDTERM EXAMINATION 2008-10-23/5:15-6:45 REC-200, EVI-350, RCH-106, HH-139

ECE 250 Data Structures and Algorithms MIDTERM EXAMINATION 2008-10-23/5:15-6:45 REC-200, EVI-350, RCH-106, HH-139 Instructions: No aides. Turn off all electronic media and store them under your desk. If

### Persistent Binary Search Trees

Persistent Binary Search Trees Datastructures, UvA. May 30, 2008 0440949, Andreas van Cranenburgh Abstract A persistent binary tree allows access to all previous versions of the tree. This paper presents

### Lecture Notes on Binary Search Trees

Lecture Notes on Binary Search Trees 15-122: Principles of Imperative Computation Frank Pfenning Lecture 17 March 17, 2010 1 Introduction In the previous two lectures we have seen how to exploit the structure

### TREE BASIC TERMINOLOGIES

TREE Trees are very flexible, versatile and powerful non-liner data structure that can be used to represent data items possessing hierarchical relationship between the grand father and his children and

### Previous Lectures. B-Trees. External storage. Two types of memory. B-trees. Main principles

B-Trees Algorithms and data structures for external memory as opposed to the main memory B-Trees Previous Lectures Height balanced binary search trees: AVL trees, red-black trees. Multiway search trees:

### Heap. Binary Search Tree. Heaps VS BSTs. < el el. Difference between a heap and a BST:

Heaps VS BSTs Difference between a heap and a BST: Heap el Binary Search Tree el el el < el el Perfectly balanced at all times Immediate access to maximal element Easy to code Does not provide efficient

### Tree Properties (size vs height) Balanced Binary Trees

Tree Properties (size vs height) Balanced Binary Trees Due now: WA 7 (This is the end of the grace period). Note that a grace period takes the place of a late day. Now is the last time you can turn this

### CSE 326, Data Structures. Sample Final Exam. Problem Max Points Score 1 14 (2x7) 2 18 (3x6) 3 4 4 7 5 9 6 16 7 8 8 4 9 8 10 4 Total 92.

Name: Email ID: CSE 326, Data Structures Section: Sample Final Exam Instructions: The exam is closed book, closed notes. Unless otherwise stated, N denotes the number of elements in the data structure

### Simplified Analyses of Randomized Algorithms for Searching, Sorting, and Selection

Simplified Analyses of Randomized Algorithms for Searching, Sorting, and Selection i Michael T. Goodrich Roberto Tamassia Dept. of Info. & Computer Science Dept. of Computer Science University of California

### 1 2-3 Trees: The Basics

CS10: Data Structures and Object-Oriented Design (Fall 2013) November 1, 2013: 2-3 Trees: Inserting and Deleting Scribes: CS 10 Teaching Team Lecture Summary In this class, we investigated 2-3 Trees in

### , each of which contains a unique key value, say k i , R 2. such that k i equals K (or to determine that no such record exists in the collection).

The Search Problem 1 Suppose we have a collection of records, say R 1, R 2,, R N, each of which contains a unique key value, say k i. Given a particular key value, K, the search problem is to locate the

### Tables so far. set() get() delete() BST Average O(lg n) O(lg n) O(lg n) Worst O(n) O(n) O(n) RB Tree Average O(lg n) O(lg n) O(lg n)

Hash Tables Tables so far set() get() delete() BST Average O(lg n) O(lg n) O(lg n) Worst O(n) O(n) O(n) RB Tree Average O(lg n) O(lg n) O(lg n) Worst O(lg n) O(lg n) O(lg n) Table naïve array implementation

### Lecture 7: Hashing III: Open Addressing

Lecture 7: Hashing III: Open Addressing Lecture Overview Open Addressing, Probing Strategies Uniform Hashing, Analysis Cryptographic Hashing Readings CLRS Chapter.4 (and.3.3 and.5 if interested) Open Addressing

### Chapter 8: Bags and Sets

Chapter 8: Bags and Sets In the stack and the queue abstractions, the order that elements are placed into the container is important, because the order elements are removed is related to the order in which

### Lecture 1: Course overview, circuits, and formulas

Lecture 1: Course overview, circuits, and formulas Topics in Complexity Theory and Pseudorandomness (Spring 2013) Rutgers University Swastik Kopparty Scribes: John Kim, Ben Lund 1 Course Information Swastik

### Binary Search Trees. A Generic Tree. Binary Trees. Nodes in a binary search tree ( B-S-T) are of the form. P parent. Key. Satellite data L R

Binary Search Trees A Generic Tree Nodes in a binary search tree ( B-S-T) are of the form P parent Key A Satellite data L R B C D E F G H I J The B-S-T has a root node which is the only node whose parent

### Sequences in the C++ STL

CS 311 Data Structures and Algorithms Lecture Slides Wednesday, November 4, 2009 Glenn G. Chappell Department of Computer Science University of Alaska Fairbanks CHAPPELLG@member.ams.org 2005 2009 Glenn

### Binary Heaps * * * * * * * / / \ / \ / \ / \ / \ * * * * * * * * * * * / / \ / \ / / \ / \ * * * * * * * * * *

Binary Heaps A binary heap is another data structure. It implements a priority queue. Priority Queue has the following operations: isempty add (with priority) remove (highest priority) peek (at highest

### Data Structures Using C++ 2E. Chapter 5 Linked Lists

Data Structures Using C++ 2E Chapter 5 Linked Lists Doubly Linked Lists Traversed in either direction Typical operations Initialize the list Destroy the list Determine if list empty Search list for a given

### Outline BST Operations Worst case Average case Balancing AVL Red-black B-trees. Binary Search Trees. Lecturer: Georgy Gimel farb

Binary Search Trees Lecturer: Georgy Gimel farb COMPSCI 220 Algorithms and Data Structures 1 / 27 1 Properties of Binary Search Trees 2 Basic BST operations The worst-case time complexity of BST operations

### Data storage Tree indexes

Data storage Tree indexes Rasmus Pagh February 7 lecture 1 Access paths For many database queries and updates, only a small fraction of the data needs to be accessed. Extreme examples are looking or updating

### 12 Abstract Data Types

12 Abstract Data Types 12.1 Source: Foundations of Computer Science Cengage Learning Objectives After studying this chapter, the student should be able to: Define the concept of an abstract data type (ADT).

### Sources: On the Web: Slides will be available on:

C programming Introduction The basics of algorithms Structure of a C code, compilation step Constant, variable type, variable scope Expression and operators: assignment, arithmetic operators, comparison,

### Introduction to Data Structures

Introduction to Data Structures Albert Gural October 28, 2011 1 Introduction When trying to convert from an algorithm to the actual code, one important aspect to consider is how to store and manipulate

### DATA STRUCTURES USING C

DATA STRUCTURES USING C QUESTION BANK UNIT I 1. Define data. 2. Define Entity. 3. Define information. 4. Define Array. 5. Define data structure. 6. Give any two applications of data structures. 7. Give

### 12.0 Statistical Graphics and RNG

12.0 Statistical Graphics and RNG 1 Answer Questions Statistical Graphics Random Number Generators 12.1 Statistical Graphics 2 John Snow helped to end the 1854 cholera outbreak through use of a statistical

### Linked Lists: Implementation Sequences in the C++ STL

Linked Lists: Implementation Sequences in the C++ STL continued CS 311 Data Structures and Algorithms Lecture Slides Wednesday, April 1, 2009 Glenn G. Chappell Department of Computer Science University

### Algorithms. Margaret M. Fleck. 18 October 2010

Algorithms Margaret M. Fleck 18 October 2010 These notes cover how to analyze the running time of algorithms (sections 3.1, 3.3, 4.4, and 7.1 of Rosen). 1 Introduction The main reason for studying big-o

### Why you shouldn't use set (and what you should use instead) Matt Austern

Why you shouldn't use set (and what you should use instead) Matt Austern Everything in the standard C++ library is there for a reason, but it isn't always obvious what that reason is. The standard isn't

### Review of Hashing: Integer Keys

CSE 326 Lecture 13: Much ado about Hashing Today s munchies to munch on: Review of Hashing Collision Resolution by: Separate Chaining Open Addressing \$ Linear/Quadratic Probing \$ Double Hashing Rehashing

### The Union-Find Problem Kruskal s algorithm for finding an MST presented us with a problem in data-structure design. As we looked at each edge,

The Union-Find Problem Kruskal s algorithm for finding an MST presented us with a problem in data-structure design. As we looked at each edge, cheapest first, we had to determine whether its two endpoints

### Data Structures Fibonacci Heaps, Amortized Analysis

Chapter 4 Data Structures Fibonacci Heaps, Amortized Analysis Algorithm Theory WS 2012/13 Fabian Kuhn Fibonacci Heaps Lacy merge variant of binomial heaps: Do not merge trees as long as possible Structure:

### Lecture 6: Binary Search Trees CSCI 700 - Algorithms I. Andrew Rosenberg

Lecture 6: Binary Search Trees CSCI 700 - Algorithms I Andrew Rosenberg Last Time Linear Time Sorting Counting Sort Radix Sort Bucket Sort Today Binary Search Trees Data Structures Data structure is a

### Ordered Lists and Binary Trees

Data Structures and Algorithms Ordered Lists and Binary Trees Chris Brooks Department of Computer Science University of San Francisco Department of Computer Science University of San Francisco p.1/62 6-0:

### Learning Outcomes. COMP202 Complexity of Algorithms. Binary Search Trees and Other Search Trees

Learning Outcomes COMP202 Complexity of Algorithms Binary Search Trees and Other Search Trees [See relevant sections in chapters 2 and 3 in Goodrich and Tamassia.] At the conclusion of this set of lecture

### 16 Collection Classes

16 Collection Classes Collections are a key feature of the ROOT system. Many, if not most, of the applications you write will use collections. If you have used parameterized C++ collections or polymorphic

### Binary Search Trees > = Binary Search Trees 1. 2004 Goodrich, Tamassia

Binary Search Trees < 6 2 > = 1 4 8 9 Binary Search Trees 1 Binary Search Trees A binary search tree is a binary tree storing keyvalue entries at its internal nodes and satisfying the following property:

### Record Storage and Primary File Organization

Record Storage and Primary File Organization 1 C H A P T E R 4 Contents Introduction Secondary Storage Devices Buffering of Blocks Placing File Records on Disk Operations on Files Files of Unordered Records

### Introduction to Algorithms March 10, 2004 Massachusetts Institute of Technology Professors Erik Demaine and Shafi Goldwasser Quiz 1.

Introduction to Algorithms March 10, 2004 Massachusetts Institute of Technology 6.046J/18.410J Professors Erik Demaine and Shafi Goldwasser Quiz 1 Quiz 1 Do not open this quiz booklet until you are directed

### Data Structures Using C++ 2E. Chapter 5 Linked Lists

Data Structures Using C++ 2E Chapter 5 Linked Lists Test #1 Next Thursday During Class Cover through (near?) end of Chapter 5 Objectives Learn about linked lists Become aware of the basic properties of

### Lecture 12 Doubly Linked Lists (with Recursion)

Lecture 12 Doubly Linked Lists (with Recursion) In this lecture Introduction to Doubly linked lists What is recursion? Designing a node of a DLL Recursion and Linked Lists o Finding a node in a LL (recursively)

### 7.1 Our Current Model

Chapter 7 The Stack In this chapter we examine what is arguably the most important abstract data type in computer science, the stack. We will see that the stack ADT and its implementation are very simple.

### AS-2261 M.Sc.(First Semester) Examination-2013 Paper -fourth Subject-Data structure with algorithm

AS-2261 M.Sc.(First Semester) Examination-2013 Paper -fourth Subject-Data structure with algorithm Time: Three Hours] [Maximum Marks: 60 Note Attempts all the questions. All carry equal marks Section A

### Why Use Binary Trees?

Binary Search Trees Why Use Binary Trees? Searches are an important application. What other searches have we considered? brute force search (with array or linked list) O(N) binarysearch with a pre-sorted

### Fast Sequential Summation Algorithms Using Augmented Data Structures

Fast Sequential Summation Algorithms Using Augmented Data Structures Vadim Stadnik vadim.stadnik@gmail.com Abstract This paper provides an introduction to the design of augmented data structures that offer

### A Study on Skip Lists, Jump Lists and Random Binary. Search Trees. (Kuan-Chien Wang) (Prof. Wei-Mei Chen)

A Study on Skip Lists, Jump Lists and Random Binary Search Trees (Kuan-Chien Wang) (Prof. Wei-Mei Chen) Thesis for Master of Science Department of Applied Mathematics Tatung University July 2004 kevin_9999@hotmail.com

### Class Notes CS 3137. 1 Creating and Using a Huffman Code. Ref: Weiss, page 433

Class Notes CS 3137 1 Creating and Using a Huffman Code. Ref: Weiss, page 433 1. FIXED LENGTH CODES: Codes are used to transmit characters over data links. You are probably aware of the ASCII code, a fixed-length

### A binary heap is a complete binary tree, where each node has a higher priority than its children. This is called heap-order property

CmSc 250 Intro to Algorithms Chapter 6. Transform and Conquer Binary Heaps 1. Definition A binary heap is a complete binary tree, where each node has a higher priority than its children. This is called

### Lecture 5 - CPA security, Pseudorandom functions

Lecture 5 - CPA security, Pseudorandom functions Boaz Barak October 2, 2007 Reading Pages 82 93 and 221 225 of KL (sections 3.5, 3.6.1, 3.6.2 and 6.5). See also Goldreich (Vol I) for proof of PRF construction.

### Intro. to the Divide-and-Conquer Strategy via Merge Sort CMPSC 465 CLRS Sections 2.3, Intro. to and various parts of Chapter 4

Intro. to the Divide-and-Conquer Strategy via Merge Sort CMPSC 465 CLRS Sections 2.3, Intro. to and various parts of Chapter 4 I. Algorithm Design and Divide-and-Conquer There are various strategies we

### Binary Search. Search for x in a sorted array A.

Divide and Conquer A general paradigm for algorithm design; inspired by emperors and colonizers. Three-step process: 1. Divide the problem into smaller problems. 2. Conquer by solving these problems. 3.

### PES Institute of Technology-BSC QUESTION BANK

PES Institute of Technology-BSC Faculty: Mrs. R.Bharathi CS35: Data Structures Using C QUESTION BANK UNIT I -BASIC CONCEPTS 1. What is an ADT? Briefly explain the categories that classify the functions

### In our lecture about Binary Search Trees (BST) we have seen that the operations of searching for and inserting an element in a BST can be of the

In our lecture about Binary Search Trees (BST) we have seen that the operations of searching for and inserting an element in a BST can be of the order O(log n) (in the best case) and of the order O(n)

### Balanced Binary Search Tree

AVL Trees / Slide 1 Balanced Binary Search Tree Worst case height of binary search tree: N-1 Insertion, deletion can be O(N) in the worst case We want a tree with small height Height of a binary tree with

### Two Parts. Filesystem Interface. Filesystem design. Interface the user sees. Implementing the interface

File Management Two Parts Filesystem Interface Interface the user sees Organization of the files as seen by the user Operations defined on files Properties that can be read/modified Filesystem design Implementing

### Analysis of Algorithms I: Binary Search Trees

Analysis of Algorithms I: Binary Search Trees Xi Chen Columbia University Hash table: A data structure that maintains a subset of keys from a universe set U = {0, 1,..., p 1} and supports all three dictionary

### C programming: exercise sheet L2-STUE (2011-2012)

C programming: exercise sheet L2-STUE (2011-2012) Algorithms and Flowcharts Exercise 1: comparison Write the flowchart and associated algorithm that compare two numbers a and b. Exercise 2: 2 nd order

### A binary search tree is a binary tree with a special property called the BST-property, which is given as follows:

Chapter 12: Binary Search Trees A binary search tree is a binary tree with a special property called the BST-property, which is given as follows: For all nodes x and y, if y belongs to the left subtree

### Data Structures and Algorithms

Data Structures and Algorithms CS245-2016S-06 Binary Search Trees David Galles Department of Computer Science University of San Francisco 06-0: Ordered List ADT Operations: Insert an element in the list

### UNIVERSITI MALAYSIA SARAWAK KOTA SAMARAHAN SARAWAK PSD2023 ALGORITHM & DATA STRUCTURE

STUDENT IDENTIFICATION NO UNIVERSITI MALAYSIA SARAWAK 94300 KOTA SAMARAHAN SARAWAK FAKULTI SAINS KOMPUTER & TEKNOLOGI MAKLUMAT (Faculty of Computer Science & Information Technology) Diploma in Multimedia

### A skip list container class in Python

A skip list container class in Python Abstract An alternative to balanced trees John W. Shipman 2012-11-29 13:23 Describes a module in the Python programming language that implements a skip list, a data

### 5. A full binary tree with n leaves contains [A] n nodes. [B] log n 2 nodes. [C] 2n 1 nodes. [D] n 2 nodes.

1. The advantage of.. is that they solve the problem if sequential storage representation. But disadvantage in that is they are sequential lists. [A] Lists [B] Linked Lists [A] Trees [A] Queues 2. The

### Data Structures. Jaehyun Park. CS 97SI Stanford University. June 29, 2015

Data Structures Jaehyun Park CS 97SI Stanford University June 29, 2015 Typical Quarter at Stanford void quarter() { while(true) { // no break :( task x = GetNextTask(tasks); process(x); // new tasks may

### S. Muthusundari. Research Scholar, Dept of CSE, Sathyabama University Chennai, India e-mail: nellailath@yahoo.co.in. Dr. R. M.

A Sorting based Algorithm for the Construction of Balanced Search Tree Automatically for smaller elements and with minimum of one Rotation for Greater Elements from BST S. Muthusundari Research Scholar,

### Vimal P.Parmar Research Scholar, Department of Computer Science Saurashtra University, Rajkot Gujarat, INDIA

Comparing Linear Search and Binary Search Algorithms to Search an Element from a Linear List Implemented through Static Array, Dynamic Array and Linked List Vimal P.Parmar Research Scholar, Department

### C++ Keywords. If/else Selection Structure. Looping Control Structures. Switch Statements. Example Program

C++ Keywords There are many keywords in C++ that are not used in other languages. bool, const_cast, delete, dynamic_cast, const, enum, extern, register, sizeof, typedef, explicit, friend, inline, mutable,

### Integer multiplication

Integer multiplication Suppose we have two unsigned integers, A and B, and we wish to compute their product. Let A be the multiplicand and B the multiplier: A n 1... A 1 A 0 multiplicand B n 1... B 1 B

### Operation Count; Numerical Linear Algebra

10 Operation Count; Numerical Linear Algebra 10.1 Introduction Many computations are limited simply by the sheer number of required additions, multiplications, or function evaluations. If floating-point

### 2. Compressing data to reduce the amount of transmitted data (e.g., to save money).

Presentation Layer The presentation layer is concerned with preserving the meaning of information sent across a network. The presentation layer may represent (encode) the data in various ways (e.g., data

### Analysis of a Search Algorithm

CSE 326 Lecture 4: Lists and Stacks 1. Agfgd 2. Dgsdsfd 3. Hdffdsf 4. Sdfgsfdg 5. Tefsdgass We will review: Analysis: Searching a sorted array (from last time) List ADT: Insert, Delete, Find, First, Kth,

### CMPS 102 Solutions to Homework 1

CMPS 0 Solutions to Homework Lindsay Brown, lbrown@soe.ucsc.edu September 9, 005 Problem..- p. 3 For inputs of size n insertion sort runs in 8n steps, while merge sort runs in 64n lg n steps. For which