# Discrete Mathematics, Chapter 3: Algorithms

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Discrete Mathematics, Chapter 3: Algorithms Richard Mayr University of Edinburgh, UK Richard Mayr (University of Edinburgh, UK) Discrete Mathematics. Chapter 3 1 / 28

2 Outline 1 Properties of Algorithms 2 The Growth of Functions 3 Complexity of Algorithms Richard Mayr (University of Edinburgh, UK) Discrete Mathematics. Chapter 3 2 / 28

3 Algorithms (Abu Ja far Mohammed Ibin Musa Al-Khowarizmi, ) Definition An algorithm is a finite set of precise instructions for performing a computation or for solving a problem. Example: Describe an algorithm for finding the maximum value in a finite sequence of integers. Description of algorithms in pseudocode: Intermediate step between English prose and formal coding in a programming language. Focus on the fundamental operation of the program, instead of peculiarities of a given programming language. Analyze the time required to solve a problem using an algorithm, independent of the actual programming language. Richard Mayr (University of Edinburgh, UK) Discrete Mathematics. Chapter 3 3 / 28

4 Properties of Algorithms Input: An algorithm has input values from a specified set. Output: From the input values, the algorithm produces the output values from a specified set. The output values are the solution. Correctness: An algorithm should produce the correct output values for each set of input values. Finiteness: An algorithm should produce the output after a finite number of steps for any input. Effectiveness: It must be possible to perform each step of the algorithm correctly and in a finite amount of time. Generality: The algorithm should work for all problems of the desired form. Richard Mayr (University of Edinburgh, UK) Discrete Mathematics. Chapter 3 4 / 28

5 Example: Linear Search Prose: Locate an item in a list by examining the sequence of list elements one at a time, starting at the beginning. More formal prose: Find item x in the list [a 1, a 2,..., a n ]. First compare x with a 1. If they are equal, return the position 1. If not, try a 2. If x = a 2, return the position 2. Keep going, and if no match is found when the entire list is scanned, return 0. Pseudocode: Algorithm 1: Linear Search Input: x : integer, [a 1,..., a n ] : list of distinct integers Output: Index i s.t. x = a i or 0 if x is not in the list. i := 1; while i n and x a i do i := i + 1; if i n then result := i else result := 0; return result; Richard Mayr (University of Edinburgh, UK) Discrete Mathematics. Chapter 3 5 / 28

6 Binary Search Prose description: Assume the input is a list of items in increasing order, and the target element to be found. The algorithm begins by comparing the target with the middle element. If the middle element is strictly lower than the target, then the search proceeds with the upper half of the list. Otherwise, the search proceeds with the lower half of the list (including the middle). Repeat this process until we have a list of size 1. If target is equal to the single element in the list, then the position is returned. Otherwise, 0 is returned to indicate that the element was not found. Richard Mayr (University of Edinburgh, UK) Discrete Mathematics. Chapter 3 6 / 28

7 Binary Search Pseudocode: Algorithm 2: Binary Search Input: x : integer, [a 1,..., a n ] : strictly increasing list of integers Output: Index i s.t. x = a i or 0 if x is not in the list. i := 1; // i is the left endpoint of the interval j := n; // j is the right endpoint of the interval while i < j do m := (i + j)/2 ; if x > a m then i := m + 1 else j := m; if x = a i then result := i else result := 0; return result; Richard Mayr (University of Edinburgh, UK) Discrete Mathematics. Chapter 3 7 / 28

8 Example: Binary Search Find target 19 in the list: The list has 16 elements, so the midpoint is 8. The value in the 8th position is 10. As 19>10, search is restricted to positions The midpoint of the list (positions 9 through 16) is now the 12th position with a value of 16. Since 19 > 16, further search is restricted to the 13th position and above The midpoint of the current list is now the 14th position with a value of 19. Since 19 19, further search is restricted to the portion from the 13th through the 14th positions The midpoint of the current list is now the 13th position with a value of 18. Since 19 > 18, search is restricted to position Now the list has a single element and the loop ends. Since 19 = 19, the location 14 is returned. Richard Mayr (University of Edinburgh, UK) Discrete Mathematics. Chapter 3 8 / 28

9 Greedy Algorithms Optimization problems minimize or maximize some parameter over all possible inputs. Examples of optimization problems: Finding a route between two cities with the smallest total mileage. Determining how to encode messages using the fewest possible bits. Finding the fiber links between network nodes using the least amount of fiber. Optimization problems can often be solved using a greedy algorithm, which makes the best (by a local criterion) choice at each step. This does not necessarily produce an optimal solution to the overall problem, but in many instances, it does. After specifying what the best choice at each step is, we try to prove that this approach always produces an optimal solution, or find a counterexample to show that it does not. Richard Mayr (University of Edinburgh, UK) Discrete Mathematics. Chapter 3 9 / 28

10 Example: Greedy Scheduling We have a group of proposed talks with start and end times. Construct a greedy algorithm to schedule as many as possible in a lecture hall, under the following assumptions: When a talk starts, it continues till the end. (Indivisible). No two talks can occur at the same time. (Mutually exclusive.) A talk can begin at the same time that another ends. Once we have selected some of the talks, we cannot add a talk which is incompatible with those already selected because it overlaps at least one of these previously selected talks. How should we make the best choice at each step of the algorithm? That is, which talk do we pick? The talk that starts earliest among those compatible with already chosen talks? The talk that is shortest among those already compatible? The talk that ends earliest among those compatible with already chosen talks? Richard Mayr (University of Edinburgh, UK) Discrete Mathematics. Chapter 3 10 / 28

11 Greedy Scheduling Picking the shortest talk doesn t work. But picking the one that ends soonest does work. The algorithm is specified on the next page. Richard Mayr (University of Edinburgh, UK) Discrete Mathematics. Chapter 3 11 / 28

12 A Greedy Scheduling Algorithm At each step, choose the talks with the earliest ending time among the talks compatible with those selected. Algorithm 3: Greedy Scheduling by End Time Input: s 1, s 2,..., s n start times and e 1, e 2,..., e n end times Output: An optimal set S {1,..., n} of talks to be scheduled. Sort talks by end time and reorder so that e 1 e 2 e n S := ; for j := 1 to n do if Talk j is compatible with S then S := S {j} return S; Note: Scheduling problems appear in many applications. Many of them (unlike this simple one) are NP-complete and do not allow efficient greedy algorithms. Richard Mayr (University of Edinburgh, UK) Discrete Mathematics. Chapter 3 12 / 28

13 The Growth of Functions Given functions f : N R or f : R R. Analyzing how fast a function grows. Comparing two functions. Comparing the efficiently of different algorithms that solve the same problem. Applications in number theory (Chapter 4) and combinatorics (Chapters 6 and 8). Richard Mayr (University of Edinburgh, UK) Discrete Mathematics. Chapter 3 13 / 28

14 Big-O Notation Definition Let f, g : R R. We say that f is O(g) if there are constants C and k such that x > k. f (x) C g(x) This is read as f is big-o of g or g asymptotically dominates f. The constants C and k are called witnesses to the relationship between f and g. Only one pair of witnesses is needed. (One pair implies many pairs, since one can always make k or C larger.) Common abuses of notation: Often one finds this written as f (x) is big-o of g(x) or f (x) = O(g(x)). This is not strictly true, since big-o refers to functions and not their values, and the equality does not hold. Strictly speaking O(g) is the class of all functions f that satisfy the condition above. So it would be formally correct to write f O(g). Richard Mayr (University of Edinburgh, UK) Discrete Mathematics. Chapter 3 14 / 28

15 Illustration of Big-O Notation f (x) = x 2 + 2x + 1, g(x) = x 2. f is O(g) witness k = 1 and C = 4. Abusing notation, this is often written as f (x) = x 2 + 2x + 1 is O(x 2 ). Richard Mayr (University of Edinburgh, UK) Discrete Mathematics. Chapter 3 15 / 28

16 Properties of Big-O Notation If f is O(g) and g is O(f ) then one says that f and g are of the same order. If f is O(g) and h(x) g(x) for all positive real numbers x then f is O(h). The O-notation describes upper bounds on how fast functions grow. E.g., f (x) = x 2 + 3x is O(x 2 ) but also O(x 3 ), etc. Often one looks for a simple function g that is as small as possible such that still f is O(g). (The word simple is important, since trivially f is O(f ).) Richard Mayr (University of Edinburgh, UK) Discrete Mathematics. Chapter 3 16 / 28

17 Example Bounds on functions. Prove that f (x) = a n x n + a n 1 x n a 1 x + a 0 is O(x n ) n is O(n 2 ). n! = 1 2 n is O(n n ). log(n!) is O(n log(n)). Richard Mayr (University of Edinburgh, UK) Discrete Mathematics. Chapter 3 17 / 28

18 Growth of Common Functions Richard Mayr (University of Edinburgh, UK) Discrete Mathematics. Chapter 3 18 / 28

19 Useful Big-O Estimates If d > c > 1, then n c is O(n d ), but n d is not O(n c ). If b > 1 and c and d are positive, then (log b n) c is O(n d ), but n d is not O((log b n) c ). If b > 1 and d is positive, then n d is O(b n ), but b n is not O(n d ). If c > b > 1, then b n is O(c n ), but c n is not O(b n ). If f 1 (x) is O(g 1 (x)) and f 2 (x) is O(g 2 (x)) then (f 1 + f 2 )(x) is O(max( g 1 (x), g 2 (x) )). If f 1 is O(g 1 ) and f 2 is O(g 2 ) then (f 1 f 2 ) is O(g 1 g 2 ). Note: These estimates are very important for analyzing algorithms. Suppose that g(n) = 5n 2 + 7n 3 and f is a very complex function that you cannot determine exactly, but you know that f is O(n 3 ). Then you can still derive that n f (n) is O(n 4 ) and g(f (n)) is O(n 6 ). Richard Mayr (University of Edinburgh, UK) Discrete Mathematics. Chapter 3 19 / 28

20 Big-Omega Notation Definition Let f, g : R R. We say that f is Ω(g) if there are constants C and k such that x > k. f (x) C g(x) This is read as f is big-omega of g. The constants C and k are called witnesses to the relationship between f and g. Big-O gives an upper bound on the growth of a function, while Big-Omega gives a lower bound. Big-Omega tells us that a function grows at least as fast as another. Similar abuse of notation as for big-o. f is Ω(g) if and only if g is O(f ). (Prove this by using the definitions of O and Ω.) Richard Mayr (University of Edinburgh, UK) Discrete Mathematics. Chapter 3 20 / 28

21 Big-Theta Notation Definition Let f, g : R R. We say that f is Θ(g) if f is O(g) and f is Ω(g). We say that f is big-theta of g and also that f is of order g and also that f and g are of the same order. f is Θ(g) if and only if there exists constants C 1, C 2 and k such that C 1 g(x) < f (x) < C 2 g(x) if x > k. This follows from the definitions of big-o and big-omega. Richard Mayr (University of Edinburgh, UK) Discrete Mathematics. Chapter 3 21 / 28

22 Example Show that the sum n of the first n positive integers is Θ(n 2 ). Solution: Let f (n) = n. We have previously shown that f (n) is O(n 2 ). To show that f (n) is Ω(n 2 ), we need a positive constant C such that f (n) > Cn 2 for sufficiently large n. Summing only the terms greater than n/2 we obtain the inequality n n/2 + ( n/2 + 1) + + n n/2 + n/2 + + n/2 = (n n/2 + 1) n/2 (n/2)(n/2) = n 2 /4 Taking C = 1/4, f (n) > Cn 2 for all positive integers n. Hence, f (n) is Ω(n 2 ), and we can conclude that f (n) is Θ(n 2 ). Richard Mayr (University of Edinburgh, UK) Discrete Mathematics. Chapter 3 22 / 28

23 Complexity of Algorithms Given an algorithm, how efficient is this algorithm for solving a problem given input of a particular size? How much time does this algorithm use to solve a problem? How much computer memory does this algorithm use to solve a problem? We measure time complexity in terms of the number of operations an algorithm uses and use big-o and big-theta notation to estimate the time complexity. Compare the efficiency of different algorithms for the same problem. We focus on the worst-case time complexity of an algorithm. Derive an upper bound on the number of operations an algorithm uses to solve a problem with input of a particular size. (As opposed to the average-case complexity.) Here: Ignore implementation details and hardware properties. See courses on algorithms and complexity. Richard Mayr (University of Edinburgh, UK) Discrete Mathematics. Chapter 3 23 / 28

24 Worst-Case Complexity of Linear Search Algorithm 4: Linear Search Input: x : integer, [a 1,..., a n ] : list of distinct integers Output: Index i s.t. x = a i or 0 if x is not in the list. i := 1; while i n and x a i do i := i + 1; if i n then result := i else result := 0; return result; Count the number of comparisons. At each step two comparisons are made; i n and x a i. To end the loop, one comparison i n is made. After the loop, one more i n comparison is made. If x = a i, 2i + 1 comparisons are used. If x is not on the list, 2n + 1 comparisons are made and then an additional comparison is used to exit the loop. So, in the worst case 2n + 2 comparisons are made. Hence, the complexity is Θ(n). Richard Mayr (University of Edinburgh, UK) Discrete Mathematics. Chapter 3 24 / 28

25 Average-Case Complexity of Linear Search For many problems, determining the average-case complexity is very difficult. (And often not very useful, since the real distribution of input cases does not match the assumptions.) However, for linear search the average-case is easy. Assume the element is in the list and that the possible positions are equally likely. By the argument on the previous slide, if x = a i, the number of comparisons is 2i + 1. Hence, the average-case complexity of linear search is 1 n 2i + 1 = n + 2 n Which is Θ(n). i=1 Richard Mayr (University of Edinburgh, UK) Discrete Mathematics. Chapter 3 25 / 28

26 Worst-Case Complexity of Binary Search Algorithm 5: Binary Search Input: x : integer, [a 1,..., a n ] : strictly increasing list of integers Output: Index i s.t. x = a i or 0 if x is not in the list. i := 1; // i is the left endpoint of the interval j := n; // j is the right endpoint of the interval while i < j do m := (i + j)/2 ; if x > a m then i := m + 1 else j := m; if x = a i then result := i else result := 0; return result; Assume (for simplicity) n = 2 k elements. Note that k = log n. Two comparisons are made at each stage; i < j, and x > a m. At the first iteration the size of the list is 2 k and after the first iteration it is 2 k 1. Then 2 k 2 and so on until the size of the list is 2 1 = 2. At the last step, a comparison tells us that the size of the list is the size is 2 0 = 1 and the element is compared with the single remaining element. Hence, at most 2k + 2 = 2 log n + 2 comparisons are made. Θ(log n). Richard Mayr (University of Edinburgh, UK) Discrete Mathematics. Chapter 3 26 / 28

27 Terminology for the Complexity of Algorithms Richard Mayr (University of Edinburgh, UK) Discrete Mathematics. Chapter 3 27 / 28

28 Further topics See courses on algorithms and complexity for Space vs. time complexity Intractable problems Complexity classes: E.g., P, NP, PSPACE, EXPTIME, EXPSPACE, etc. Undecidable problems and the limits of algorithms. Richard Mayr (University of Edinburgh, UK) Discrete Mathematics. Chapter 3 28 / 28

### Why? A central concept in Computer Science. Algorithms are ubiquitous.

Analysis of Algorithms: A Brief Introduction Why? A central concept in Computer Science. Algorithms are ubiquitous. Using the Internet (sending email, transferring files, use of search engines, online

### Algorithms and Data Structures

Algorithm Analysis Page 1 BFH-TI: Softwareschule Schweiz Algorithm Analysis Dr. CAS SD01 Algorithm Analysis Page 2 Outline Course and Textbook Overview Analysis of Algorithm Pseudo-Code and Primitive Operations

### Today s Outline. Exercise. Binary Search Analysis. Linear Search Analysis. Asymptotic Analysis. Analyzing Code. Announcements. Asymptotic Analysis

Today s Outline Announcements Assignment #1 due Thurs, Oct 7 at 11:45pm Asymptotic Analysis Asymptotic Analysis CSE 7 Data Structures & Algorithms Ruth Anderson Autumn 2010 Exercise Analyzing Code bool

### In mathematics, it is often important to get a handle on the error term of an approximation. For instance, people will write

Big O notation (with a capital letter O, not a zero), also called Landau's symbol, is a symbolism used in complexity theory, computer science, and mathematics to describe the asymptotic behavior of functions.

### 6.080 / 6.089 Great Ideas in Theoretical Computer Science Spring 2008

MIT OpenCourseWare http://ocw.mit.edu 6.080 / 6.089 Great Ideas in Theoretical Computer Science Spring 008 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

### CS/COE 1501 http://cs.pitt.edu/~bill/1501/

CS/COE 1501 http://cs.pitt.edu/~bill/1501/ Lecture 01 Course Introduction Meta-notes These notes are intended for use by students in CS1501 at the University of Pittsburgh. They are provided free of charge

### Analysis of Algorithms: The Non-recursive Case

Analysis of Algorithms: The Non-recursive Case Key topics: * Introduction * Generalizing Running Time * Doing a Timing Analysis * Big-Oh Notation * Big-Oh Operations * Analyzing Some Simple Programs -

### 2.3 Scheduling jobs on identical parallel machines

2.3 Scheduling jobs on identical parallel machines There are jobs to be processed, and there are identical machines (running in parallel) to which each job may be assigned Each job = 1,,, must be processed

### Efficiency of algorithms. Algorithms. Efficiency of algorithms. Binary search and linear search. Best, worst and average case.

Algorithms Efficiency of algorithms Computational resources: time and space Best, worst and average case performance How to compare algorithms: machine-independent measure of efficiency Growth rate Complexity

### CSE373: Data Structures and Algorithms Lecture 3: Math Review; Algorithm Analysis. Linda Shapiro Winter 2015

CSE373: Data Structures and Algorithms Lecture 3: Math Review; Algorithm Analysis Linda Shapiro Today Registration should be done. Homework 1 due 11:59 pm next Wednesday, January 14 Review math essential

### Algorithms, Integers

CHAPTER 3 Algorithms, Integers 3.1. Algorithms Consider the following list of instructions to find the maximum of three numbers a, b, c: 1. Assign variable x the value of a. 2. If b > x then assign x the

### Cost Model: Work, Span and Parallelism. 1 The RAM model for sequential computation:

CSE341T 08/31/2015 Lecture 3 Cost Model: Work, Span and Parallelism In this lecture, we will look at how one analyze a parallel program written using Cilk Plus. When we analyze the cost of an algorithm

### Lecture 4: Properties of and Rules for

Lecture 4: Properties of and Rules for Asymptotic Big-Oh, Big-Omega, and Big-Theta Notation Georgy Gimel farb COMPSCI 220 Algorithms and Data Structures 1 / 13 1 Time complexity 2 Big-Oh rules Scaling

### Class Overview. CSE 326: Data Structures. Goals. Goals. Data Structures. Goals. Introduction

Class Overview CSE 326: Data Structures Introduction Introduction to many of the basic data structures used in computer software Understand the data structures Analyze the algorithms that use them Know

### CSC148 Lecture 8. Algorithm Analysis Binary Search Sorting

CSC148 Lecture 8 Algorithm Analysis Binary Search Sorting Algorithm Analysis Recall definition of Big Oh: We say a function f(n) is O(g(n)) if there exists positive constants c,b such that f(n)

### Chapter 3. if 2 a i then location: = i. Page 40

Chapter 3 1. Describe an algorithm that takes a list of n integers a 1,a 2,,a n and finds the number of integers each greater than five in the list. Ans: procedure greaterthanfive(a 1,,a n : integers)

### Intro. to the Divide-and-Conquer Strategy via Merge Sort CMPSC 465 CLRS Sections 2.3, Intro. to and various parts of Chapter 4

Intro. to the Divide-and-Conquer Strategy via Merge Sort CMPSC 465 CLRS Sections 2.3, Intro. to and various parts of Chapter 4 I. Algorithm Design and Divide-and-Conquer There are various strategies we

### Analysis of Binary Search algorithm and Selection Sort algorithm

Analysis of Binary Search algorithm and Selection Sort algorithm In this section we shall take up two representative problems in computer science, work out the algorithms based on the best strategy to

### CMPS 102 Solutions to Homework 1

CMPS 0 Solutions to Homework Lindsay Brown, lbrown@soe.ucsc.edu September 9, 005 Problem..- p. 3 For inputs of size n insertion sort runs in 8n steps, while merge sort runs in 64n lg n steps. For which

### Lecture 3: Finding integer solutions to systems of linear equations

Lecture 3: Finding integer solutions to systems of linear equations Algorithmic Number Theory (Fall 2014) Rutgers University Swastik Kopparty Scribe: Abhishek Bhrushundi 1 Overview The goal of this lecture

### Algorithms. Margaret M. Fleck. 18 October 2010

Algorithms Margaret M. Fleck 18 October 2010 These notes cover how to analyze the running time of algorithms (sections 3.1, 3.3, 4.4, and 7.1 of Rosen). 1 Introduction The main reason for studying big-o

### Problem Set 7 Solutions

8 8 Introduction to Algorithms May 7, 2004 Massachusetts Institute of Technology 6.046J/18.410J Professors Erik Demaine and Shafi Goldwasser Handout 25 Problem Set 7 Solutions This problem set is due in

### 3. Mathematical Induction

3. MATHEMATICAL INDUCTION 83 3. Mathematical Induction 3.1. First Principle of Mathematical Induction. Let P (n) be a predicate with domain of discourse (over) the natural numbers N = {0, 1,,...}. If (1)

### The Union-Find Problem Kruskal s algorithm for finding an MST presented us with a problem in data-structure design. As we looked at each edge,

The Union-Find Problem Kruskal s algorithm for finding an MST presented us with a problem in data-structure design. As we looked at each edge, cheapest first, we had to determine whether its two endpoints

### Introduction to Algorithms Review information for Prelim 1 CS 4820, Spring 2010 Distributed Wednesday, February 24

Introduction to Algorithms Review information for Prelim 1 CS 4820, Spring 2010 Distributed Wednesday, February 24 The final exam will cover seven topics. 1. greedy algorithms 2. divide-and-conquer algorithms

### Zabin Visram Room CS115 CS126 Searching. Binary Search

Zabin Visram Room CS115 CS126 Searching Binary Search Binary Search Sequential search is not efficient for large lists as it searches half the list, on average Another search algorithm Binary search Very

### Induction. Margaret M. Fleck. 10 October These notes cover mathematical induction and recursive definition

Induction Margaret M. Fleck 10 October 011 These notes cover mathematical induction and recursive definition 1 Introduction to induction At the start of the term, we saw the following formula for computing

### Near Optimal Solutions

Near Optimal Solutions Many important optimization problems are lacking efficient solutions. NP-Complete problems unlikely to have polynomial time solutions. Good heuristics important for such problems.

### Theory of Computation Prof. Kamala Krithivasan Department of Computer Science and Engineering Indian Institute of Technology, Madras

Theory of Computation Prof. Kamala Krithivasan Department of Computer Science and Engineering Indian Institute of Technology, Madras Lecture No. # 31 Recursive Sets, Recursively Innumerable Sets, Encoding

### Approximation Algorithms

Approximation Algorithms or: How I Learned to Stop Worrying and Deal with NP-Completeness Ong Jit Sheng, Jonathan (A0073924B) March, 2012 Overview Key Results (I) General techniques: Greedy algorithms

### Classification - Examples

Lecture 2 Scheduling 1 Classification - Examples 1 r j C max given: n jobs with processing times p 1,...,p n and release dates r 1,...,r n jobs have to be scheduled without preemption on one machine taking

### COMP 250 Fall 2012 lecture 2 binary representations Sept. 11, 2012

Binary numbers The reason humans represent numbers using decimal (the ten digits from 0,1,... 9) is that we have ten fingers. There is no other reason than that. There is nothing special otherwise about

### Analysis of Computer Algorithms. Algorithm. Algorithm, Data Structure, Program

Analysis of Computer Algorithms Hiroaki Kobayashi Input Algorithm Output 12/13/02 Algorithm Theory 1 Algorithm, Data Structure, Program Algorithm Well-defined, a finite step-by-step computational procedure

### Introduction to computer science

Introduction to computer science Michael A. Nielsen University of Queensland Goals: 1. Introduce the notion of the computational complexity of a problem, and define the major computational complexity classes.

### Pigeonhole Principle Solutions

Pigeonhole Principle Solutions 1. Show that if we take n + 1 numbers from the set {1, 2,..., 2n}, then some pair of numbers will have no factors in common. Solution: Note that consecutive numbers (such

### The Goldberg Rao Algorithm for the Maximum Flow Problem

The Goldberg Rao Algorithm for the Maximum Flow Problem COS 528 class notes October 18, 2006 Scribe: Dávid Papp Main idea: use of the blocking flow paradigm to achieve essentially O(min{m 2/3, n 1/2 }

### Recursive Algorithms. Recursion. Motivating Example Factorial Recall the factorial function. { 1 if n = 1 n! = n (n 1)! if n > 1

Recursion Slides by Christopher M Bourke Instructor: Berthe Y Choueiry Fall 007 Computer Science & Engineering 35 Introduction to Discrete Mathematics Sections 71-7 of Rosen cse35@cseunledu Recursive Algorithms

### Lecture 4 Representing Data on the Computer. Ramani Duraiswami AMSC/CMSC 662 Fall 2009

Lecture 4 Representing Data on the Computer Ramani Duraiswami AMSC/CMSC 662 Fall 2009 x = ±(1+f) 2 e 0 f < 1 f = (integer < 2 52 )/ 2 52-1022 e 1023 e = integer Effects of floating point Effects of floating

### SIMS 255 Foundations of Software Design. Complexity and NP-completeness

SIMS 255 Foundations of Software Design Complexity and NP-completeness Matt Welsh November 29, 2001 mdw@cs.berkeley.edu 1 Outline Complexity of algorithms Space and time complexity ``Big O'' notation Complexity

### Divide and Conquer. Textbook Reading Chapters 4, 7 & 33.4

Divide d Conquer Textook Reading Chapters 4, 7 & 33.4 Overview Design principle Divide d conquer Proof technique Induction, induction, induction Analysis technique Recurrence relations Prolems Sorting

### Classification - Examples -1- 1 r j C max given: n jobs with processing times p 1,..., p n and release dates

Lecture 2 Scheduling 1 Classification - Examples -1-1 r j C max given: n jobs with processing times p 1,..., p n and release dates r 1,..., r n jobs have to be scheduled without preemption on one machine

### ! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. !-approximation algorithm.

Approximation Algorithms Chapter Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of

### , each of which contains a unique key value, say k i , R 2. such that k i equals K (or to determine that no such record exists in the collection).

The Search Problem 1 Suppose we have a collection of records, say R 1, R 2,, R N, each of which contains a unique key value, say k i. Given a particular key value, K, the search problem is to locate the

### (x + a) n = x n + a Z n [x]. Proof. If n is prime then the map

22. A quick primality test Prime numbers are one of the most basic objects in mathematics and one of the most basic questions is to decide which numbers are prime (a clearly related problem is to find

### 8.1 Makespan Scheduling

600.469 / 600.669 Approximation Algorithms Lecturer: Michael Dinitz Topic: Dynamic Programing: Min-Makespan and Bin Packing Date: 2/19/15 Scribe: Gabriel Kaptchuk 8.1 Makespan Scheduling Consider an instance

### CSC 180 H1F Algorithm Runtime Analysis Lecture Notes Fall 2015

1 Introduction These notes introduce basic runtime analysis of algorithms. We would like to be able to tell if a given algorithm is time-efficient, and to be able to compare different algorithms. 2 Linear

### CS 103X: Discrete Structures Homework Assignment 3 Solutions

CS 103X: Discrete Structures Homework Assignment 3 s Exercise 1 (20 points). On well-ordering and induction: (a) Prove the induction principle from the well-ordering principle. (b) Prove the well-ordering

### Information Theory and Coding Prof. S. N. Merchant Department of Electrical Engineering Indian Institute of Technology, Bombay

Information Theory and Coding Prof. S. N. Merchant Department of Electrical Engineering Indian Institute of Technology, Bombay Lecture - 17 Shannon-Fano-Elias Coding and Introduction to Arithmetic Coding

### 1 if 1 x 0 1 if 0 x 1

Chapter 3 Continuity In this chapter we begin by defining the fundamental notion of continuity for real valued functions of a single real variable. When trying to decide whether a given function is or

### Data Structures. Algorithm Performance and Big O Analysis

Data Structures Algorithm Performance and Big O Analysis What s an Algorithm? a clearly specified set of instructions to be followed to solve a problem. In essence: A computer program. In detail: Defined

### Scheduling. Open shop, job shop, flow shop scheduling. Related problems. Open shop, job shop, flow shop scheduling

Scheduling Basic scheduling problems: open shop, job shop, flow job The disjunctive graph representation Algorithms for solving the job shop problem Computational complexity of the job shop problem Open

### recursion, O(n), linked lists 6/14

recursion, O(n), linked lists 6/14 recursion reducing the amount of data to process and processing a smaller amount of data example: process one item in a list, recursively process the rest of the list

### Introduction to Algorithms March 10, 2004 Massachusetts Institute of Technology Professors Erik Demaine and Shafi Goldwasser Quiz 1.

Introduction to Algorithms March 10, 2004 Massachusetts Institute of Technology 6.046J/18.410J Professors Erik Demaine and Shafi Goldwasser Quiz 1 Quiz 1 Do not open this quiz booklet until you are directed

Approximation Algorithms Chapter Approximation Algorithms Q. Suppose I need to solve an NP-hard problem. What should I do? A. Theory says you're unlikely to find a poly-time algorithm. Must sacrifice one

### High degree graphs contain large-star factors

High degree graphs contain large-star factors Dedicated to László Lovász, for his 60th birthday Noga Alon Nicholas Wormald Abstract We show that any finite simple graph with minimum degree d contains a

CS787: Advanced Algorithms Lecture 14: Online algorithms We now shift focus to a different kind of algorithmic problem where we need to perform some optimization without knowing the input in advance. Algorithms

### Lecture Notes on Linear Search

Lecture Notes on Linear Search 15-122: Principles of Imperative Computation Frank Pfenning Lecture 5 January 29, 2013 1 Introduction One of the fundamental and recurring problems in computer science is

### Lecture 1: Course overview, circuits, and formulas

Lecture 1: Course overview, circuits, and formulas Topics in Complexity Theory and Pseudorandomness (Spring 2013) Rutgers University Swastik Kopparty Scribes: John Kim, Ben Lund 1 Course Information Swastik

### Sequential Data Structures

Sequential Data Structures In this lecture we introduce the basic data structures for storing sequences of objects. These data structures are based on arrays and linked lists, which you met in first year

### 1. Give the 16 bit signed (twos complement) representation of the following decimal numbers, and convert to hexadecimal:

Exercises 1 - number representations Questions 1. Give the 16 bit signed (twos complement) representation of the following decimal numbers, and convert to hexadecimal: (a) 3012 (b) - 435 2. For each of

### Notes on Complexity Theory Last updated: August, 2011. Lecture 1

Notes on Complexity Theory Last updated: August, 2011 Jonathan Katz Lecture 1 1 Turing Machines I assume that most students have encountered Turing machines before. (Students who have not may want to look

### ! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. #-approximation algorithm.

Approximation Algorithms 11 Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of three

### Series Convergence Tests Math 122 Calculus III D Joyce, Fall 2012

Some series converge, some diverge. Series Convergence Tests Math 22 Calculus III D Joyce, Fall 202 Geometric series. We ve already looked at these. We know when a geometric series converges and what it

### CS473 - Algorithms I

CS473 - Algorithms I Lecture 9 Sorting in Linear Time View in slide-show mode 1 How Fast Can We Sort? The algorithms we have seen so far: Based on comparison of elements We only care about the relative

### 1. Sorting (assuming sorting into ascending order) a) BUBBLE SORT

DECISION 1 Revision Notes 1. Sorting (assuming sorting into ascending order) a) BUBBLE SORT Make sure you show comparisons clearly and label each pass First Pass 8 4 3 6 1 4 8 3 6 1 4 3 8 6 1 4 3 6 8 1

### princeton univ. F 13 cos 521: Advanced Algorithm Design Lecture 6: Provable Approximation via Linear Programming Lecturer: Sanjeev Arora

princeton univ. F 13 cos 521: Advanced Algorithm Design Lecture 6: Provable Approximation via Linear Programming Lecturer: Sanjeev Arora Scribe: One of the running themes in this course is the notion of

### Operation Count; Numerical Linear Algebra

10 Operation Count; Numerical Linear Algebra 10.1 Introduction Many computations are limited simply by the sheer number of required additions, multiplications, or function evaluations. If floating-point

### ON THE COMPLEXITY OF THE GAME OF SET. {kamalika,pbg,dratajcz,hoeteck}@cs.berkeley.edu

ON THE COMPLEXITY OF THE GAME OF SET KAMALIKA CHAUDHURI, BRIGHTEN GODFREY, DAVID RATAJCZAK, AND HOETECK WEE {kamalika,pbg,dratajcz,hoeteck}@cs.berkeley.edu ABSTRACT. Set R is a card game played with a

### The Running Time of Programs

CHAPTER 3 The Running Time of Programs In Chapter 2, we saw two radically different algorithms for sorting: selection sort and merge sort. There are, in fact, scores of algorithms for sorting. This situation

### Lecture Notes on Binary Search Trees

Lecture Notes on Binary Search Trees 15-122: Principles of Imperative Computation Frank Pfenning André Platzer Lecture 17 October 23, 2014 1 Introduction In this lecture, we will continue considering associative

### 2.1 Complexity Classes

15-859(M): Randomized Algorithms Lecturer: Shuchi Chawla Topic: Complexity classes, Identity checking Date: September 15, 2004 Scribe: Andrew Gilpin 2.1 Complexity Classes In this lecture we will look

### Fairness in Routing and Load Balancing

Fairness in Routing and Load Balancing Jon Kleinberg Yuval Rabani Éva Tardos Abstract We consider the issue of network routing subject to explicit fairness conditions. The optimization of fairness criteria

### Many algorithms, particularly divide and conquer algorithms, have time complexities which are naturally

Recurrence Relations Many algorithms, particularly divide and conquer algorithms, have time complexities which are naturally modeled by recurrence relations. A recurrence relation is an equation which

### Homework Exam 1, Geometric Algorithms, 2016

Homework Exam 1, Geometric Algorithms, 2016 1. (3 points) Let P be a convex polyhedron in 3-dimensional space. The boundary of P is represented as a DCEL, storing the incidence relationships between the

### Lecture 1: Time Complexity

Computational Complexity Theory, Fall 2010 August 25 Lecture 1: Time Complexity Lecturer: Peter Bro Miltersen Scribe: Søren Valentin Haagerup 1 Introduction to the course The field of Computational Complexity

### Complexity Theory. IE 661: Scheduling Theory Fall 2003 Satyaki Ghosh Dastidar

Complexity Theory IE 661: Scheduling Theory Fall 2003 Satyaki Ghosh Dastidar Outline Goals Computation of Problems Concepts and Definitions Complexity Classes and Problems Polynomial Time Reductions Examples

### Faster deterministic integer factorisation

David Harvey (joint work with Edgar Costa, NYU) University of New South Wales 25th October 2011 The obvious mathematical breakthrough would be the development of an easy way to factor large prime numbers

### Simplified Analyses of Randomized Algorithms for Searching, Sorting, and Selection

Simplified Analyses of Randomized Algorithms for Searching, Sorting, and Selection i Michael T. Goodrich Roberto Tamassia Dept. of Info. & Computer Science Dept. of Computer Science University of California

### Sorting revisited. Build the binary search tree: O(n^2) Traverse the binary tree: O(n) Total: O(n^2) + O(n) = O(n^2)

Sorting revisited How did we use a binary search tree to sort an array of elements? Tree Sort Algorithm Given: An array of elements to sort 1. Build a binary search tree out of the elements 2. Traverse

### Guessing Game: NP-Complete?

Guessing Game: NP-Complete? 1. LONGEST-PATH: Given a graph G = (V, E), does there exists a simple path of length at least k edges? YES 2. SHORTEST-PATH: Given a graph G = (V, E), does there exists a simple

### Analysis of Algorithms, I

Analysis of Algorithms, I CSOR W4231.002 Eleni Drinea Computer Science Department Columbia University Thursday, February 26, 2015 Outline 1 Recap 2 Representing graphs 3 Breadth-first search (BFS) 4 Applications

### Lecture 3: Linear Programming Relaxations and Rounding

Lecture 3: Linear Programming Relaxations and Rounding 1 Approximation Algorithms and Linear Relaxations For the time being, suppose we have a minimization problem. Many times, the problem at hand can

### CSC2420 Fall 2012: Algorithm Design, Analysis and Theory

CSC2420 Fall 2012: Algorithm Design, Analysis and Theory Allan Borodin November 15, 2012; Lecture 10 1 / 27 Randomized online bipartite matching and the adwords problem. We briefly return to online algorithms

### Computational complexity theory

Computational complexity theory Goal: A general theory of the resources needed to solve computational problems What types of resources? Time What types of computational problems? decision problem Decision

### Integer multiplication

Integer multiplication Suppose we have two unsigned integers, A and B, and we wish to compute their product. Let A be the multiplicand and B the multiplier: A n 1... A 1 A 0 multiplicand B n 1... B 1 B

### Project and Production Management Prof. Arun Kanda Department of Mechanical Engineering Indian Institute of Technology, Delhi

Project and Production Management Prof. Arun Kanda Department of Mechanical Engineering Indian Institute of Technology, Delhi Lecture - 15 Limited Resource Allocation Today we are going to be talking about

### Divide-and-Conquer Algorithms Part Four

Divide-and-Conquer Algorithms Part Four Announcements Problem Set 2 due right now. Can submit by Monday at 2:15PM using one late period. Problem Set 3 out, due July 22. Play around with divide-and-conquer

### The Prime Numbers. Definition. A prime number is a positive integer with exactly two positive divisors.

The Prime Numbers Before starting our study of primes, we record the following important lemma. Recall that integers a, b are said to be relatively prime if gcd(a, b) = 1. Lemma (Euclid s Lemma). If gcd(a,

### Homework 5 Solutions

Homework 5 Solutions 4.2: 2: a. 321 = 256 + 64 + 1 = (01000001) 2 b. 1023 = 512 + 256 + 128 + 64 + 32 + 16 + 8 + 4 + 2 + 1 = (1111111111) 2. Note that this is 1 less than the next power of 2, 1024, which

### A Working Knowledge of Computational Complexity for an Optimizer

A Working Knowledge of Computational Complexity for an Optimizer ORF 363/COS 323 Instructor: Amir Ali Ahmadi TAs: Y. Chen, G. Hall, J. Ye Fall 2014 1 Why computational complexity? What is computational

### Loop Invariants and Binary Search

Loop Invariants and Binary Search Chapter 4.3.3 and 9.3.1-1 - Outline Ø Iterative Algorithms, Assertions and Proofs of Correctness Ø Binary Search: A Case Study - 2 - Outline Ø Iterative Algorithms, Assertions

### (Refer Slide Time: 01.26)

Discrete Mathematical Structures Dr. Kamala Krithivasan Department of Computer Science and Engineering Indian Institute of Technology, Madras Lecture # 27 Pigeonhole Principle In the next few lectures

### Factoring & Primality

Factoring & Primality Lecturer: Dimitris Papadopoulos In this lecture we will discuss the problem of integer factorization and primality testing, two problems that have been the focus of a great amount

### Best Monotone Degree Bounds for Various Graph Parameters

Best Monotone Degree Bounds for Various Graph Parameters D. Bauer Department of Mathematical Sciences Stevens Institute of Technology Hoboken, NJ 07030 S. L. Hakimi Department of Electrical and Computer

### Theoretical Computer Science (Bridging Course) Complexity

Theoretical Computer Science (Bridging Course) Complexity Gian Diego Tipaldi A scenario You are a programmer working for a logistics company Your boss asks you to implement a program that optimizes the

### What Is Recursion? Recursion. Binary search example postponed to end of lecture

Recursion Binary search example postponed to end of lecture What Is Recursion? Recursive call A method call in which the method being called is the same as the one making the call Direct recursion Recursion

### Merge Sort. 2004 Goodrich, Tamassia. Merge Sort 1

Merge Sort 7 2 9 4 2 4 7 9 7 2 2 7 9 4 4 9 7 7 2 2 9 9 4 4 Merge Sort 1 Divide-and-Conquer Divide-and conquer is a general algorithm design paradigm: Divide: divide the input data S in two disjoint subsets

### MATH 4330/5330, Fourier Analysis Section 11, The Discrete Fourier Transform

MATH 433/533, Fourier Analysis Section 11, The Discrete Fourier Transform Now, instead of considering functions defined on a continuous domain, like the interval [, 1) or the whole real line R, we wish