Why? A central concept in Computer Science. Algorithms are ubiquitous.

Size: px
Start display at page:

Download "Why? A central concept in Computer Science. Algorithms are ubiquitous."

Transcription

1 Analysis of Algorithms: A Brief Introduction Why? A central concept in Computer Science. Algorithms are ubiquitous. Using the Internet (sending , transferring files, use of search engines, online shopping). Document preparation/processing. Scheduling flights. Manufacturing. Medicine. 1

2 Algorithm: A well defined computational procedure to solve a problem. Focus: Combinatorial problems (i.e., problems whose solution space is discrete). Problem Specification: Input to the problem. Output to be produced. Problem 1: Finding the maximum value. Input: An array A[1.. n] of n integers. Output: The maximum value in the input. 2

3 Problem 2: Sorting into non-decreasing order. Input: An array A[1.. n] of n integers. Output: A permutation of the array elements so that A[1] A[2]... A[n]. Problem 3: Zero Sum. Input: An array A[1.. n] of n integers. (The integer values may be positive, negative or zero.) Output: True if the array has two elements A[i] and A[j] (i j) such that the sum A[i] + A[j] is zero; False otherwise. Note: Zero Sum is a decision problem. 3

4 Definition: Given an array A[1.. n], a block is a subarray A[i.. j], where 1 i j n. Note: Each element A[i] is a block by itself. Problem 4: Maximum Block Sum. Input: An array A[1.. n] of n integers. (The integer values may be positive, negative or zero.) Output: The maximum value among the sums of all the blocks of A. Note: Maximum Block Sum is an optimization problem. 4

5 Problem 5: Shortest Path. Input: A network G consisting of nodes and edges, with each edge having a length (non-negative number); two nodes u and v. Output: A shortest path between u and v in G. Example: a 4 b 5 7 c u 3 3 d 3 v e 2 f Note: Shortest Path is also an optimization problem. 5

6 Boolean Satisfiability Problem: Terminology and Notation: Boolean variable x: Takes on a value from {True, False}. Complement of x is denoted by x. (The symbols x and x are called literals.) Boolean operators (connectives): e.g. And (denoted by ), Or (denoted by ). A Boolean formula is constructed using literals, connectives and parentheses. 6

7 Example: The following formula F uses three Boolean variables x 1, x 2 and x 3. F = (x 1 x 2 x 3 ) (x 2 x 3 ) (x 1 x 3 ) (a) Let x 1 = True, x 2 = True and x 3 = True. assignment, F evaluates to False. For this (b) Let x 1 = True, x 2 = True and x 3 = False. For this assignment, F evaluates to True. Definition: A Boolean formula F is satisfiable if there is at least one assignment of values to the variables in F for which F evaluates to True. 7

8 Examples: Formula F = (x 1 x 2 x 3 ) (x 2 x 3 ) (x 1 x 3 ) is satisfiable. (One satisfying assignment for F is x 1 = True, x 2 = True and x 3 = False.) Formula F 1 defined by F 1 = (x 1 ) (x 2 ) (x 1 x 2 ) is not satisfiable. Problem 6: Boolean Satisfiability (SAT) Input: A formula F constructed using Boolean variables x 1, x 2,..., x n, their complements and Boolean operators. Output: True if F satisfiable and False otherwise. Note: SAT is also a decision problem. 8

9 Exercises: 1. Find the number of satisfying assignments for the formula F given on page Construct a formula F 2 using three variables x 1, x 2 and x 3 such that F 2 has exactly one satisfying assignment. Indicate the satisfying assignment. 3. Construct a Boolean formula F 3 using two variables x 1 and x 2 such that F 3 evaluates to True for every assignment. 9

10 An Algorithm for Finding the Maximum: Input: Array A[1.. n] of integers. Pseudocode: 1. Max = A[1] 2. for i = 2 to n do if (A[i] > 3. Print Max Max) then Max = A[i] Correctness of an Algorithm: Algorithm must halt for every input instance. Must produce the correct output for every input instance. 10

11 Analyzing an Algorithm: Estimating the resources (e.g. running time, memory) needed in a machine independent manner. Useful in comparing candidate algorithms for a problem. Computational Model: Each primitive operation (e.g. arithmetic operation, comparison, assignment) takes one unit of time. Running time for a specific input: Number of primitive operations executed by the algorithm for that input (expressed as a function of the input size ). 11

12 Running Time of an Algorithm: The longest running time for any input of a certain size. Also called worst-case time (or time complexity). Input Size: Depends on the problem. Examples: (a) Sorting: Number (n) of input values. (b) Graph problems: Number of nodes + Number of edges. (c) SAT problem: Number of literals + Number of operators + Number of parentheses. 12

13 Running Time Analysis Example I: Pseudocode: 1. Max = A[1] 2. for i = 2 to n do if (A[i] > Max) then Max = A[i] 3. Print Max Analysis: Input size = n. Step 1: No. of operations = 1. Step 2: The for loop executes n 1 times. Each time through the loop, we have: 13

14 (a) Two operations (comparison and increment) on i. (b) At most two operations (a comparison and an assignment) for the if statement. So, the total number of operations over all the iterations of the loop is at most 4(n 1). Step 3: No. of operations = 1. So, the total number of operations carried out by the algorithm is at most 1 + 4(n 1) + 1 = 4n 2. Running Time Analysis Example 2: Zero Sum: Input: Array A[1.. n] containing n integers. 14

15 Pseudocode: 1. for i = 1 to n 1 do 1.1 for j = i + 1 to n do if (A[i] + A[j] = 0) then Print True and stop. 2. Print False. Analysis: Step 1: The for loop runs n 1 times. For each iteration of this loop: (a) The for loop in Step 1.1 runs n i times. (b) During each iteration of this inner for loop, at most 6 operations are carried out: 15

16 Comparison and increment for j, the sum A[i] + A[j] and its comparison with 0 and print and stop operations. (c) So, the total number of operations carried out during all iterations of the inner for loop is at most 6(n i). So, the total number of operations carried out during all the iterations of the outer for loop is n 1 6(n i) = 3n(n 1). Step 2: No. of operations = 1. Conclusion: i=1 The total number of operations carried out by the algorithm is at most 3n(n 1)

17 Disadvantages: This type of detailed analysis is too tedious. The exact number of operations is not too insightful. Simplified Representation: Order or Big-O Notation. Running time as the input size becomes large (also called asymptotic running time). Facilitates the comparison of algorithms for a problem. Basic Ideas: Use only the most dominant term in the expression for the number of operations. Suppress additive and multiplicative constants. 17

18 Examples: The number of steps used by the algorithm for finding the maximum is at most 4n 2. This is expressed as O(n). So, the running time of the maximum finding algorithm is O(n). The number of steps used by the algorithm for the Zero Sum problem is at most 3n(n 1) + 1. So, the running time of the algorithm for the Zero Sum problem is O(n 2 ). Additional Examples: (a) Let f(n) = n n Then, f(n) = O(n 3 ). (b) Let g(n) = 2n log 2 n + 31n Then, g(n) = O(n log 2 n). Note: O(1) denotes a constant. 18

19 Exercises: n 1. Find the big-o representation for i=1 (2i2 ). 2. Suppose f(n) = 8n 3 + 2n 2 17 and g(n) = n 5 + n Find the big-o representations for f(n) + g(n) and f(n) g(n). Algorithms for the Maximum Block Sum Problem: Input: An array A[1.. n] of integers. Idea behind Algorithm I: Consider each block of A and compute its sum. Output the largest sum found. 19

20 Pseudocode for Algorithm I: 1. MaxSum = A[1] 2. for i = 1 to n do 2.1 for j = i to n do temp = FindSum(A, i, j) if (temp > MaxSum) then MaxSum = temp 3. Print MaxSum function FindSum(A, i, j) 1. sum = 0 2. for k = i to j do sum = sum + A[k] 3. return sum 20

21 Analysis: Each call to FindSum takes O(n) time. Each iteration of the loop in Step 2.1 runs in O(n) time (since the dominant time is due to the call to FindSum). Since the loop itself runs at most n times, the running time of the loop in Step 2.1 is O(n 2 ). The loop in Step 2 runs n times. Each iteration of this loop executes the loop in Step 2.1. Since the latter takes O(n 2 ) time, the time to complete Step 2 is O(n n 2 ) = O(n 3 ). Steps 1 and 3 take O(1) time. So, the overall running time is O(n 3 ). 21

22 Algorithm II for Maximum Block Sum Idea: For each i, (1 i n), there are n i+1 blocks whose starting point is A[i]. For each such block, Algorithm I computes the sum in O(n) time (using the FindSum function). So, the time used to compute the sums for all the blocks with starting point A[i] is O(n 2 ). It is possible to compute the sums of all the blocks that start at A[i] in O(n) time as follows. Let S i,j denote the sum of the block A[i.. j]. S i,i = A[i] S i,i+1 = S i,i + A[i + 1] S i,i+2 = S i,i+1 + A[i + 2]. 22

23 S i,n = S i,n 1 + A[n] The resulting algorithm runs in time O(n 2 ). Exercise: Write pseudocode for Algorithm II using the above idea. Verify that the running time of the algorithm is O(n 2 ). Algorithm III for Maximum Block Sum Idea: (a) Observation: A block with the maximum sum ends at A[i] for some i, 1 i n. (b) For each i, compute and store the maximum sum among all blocks that end at A[i]. (c) The largest sum found in (b) is the solution. 23

24 How to carry out Step (b): Use an auxiliary array B[1.. n]; for each i, let B[i] store the maximum sum among the blocks that end at A[i]. Note that B[1] = A[1]. For any i 2, B[i] = B[i 1] + A[i] if B[i 1] > 0 = A[i] otherwise. Example: To be presented in class. 24

25 Pseudocode for Algorithm III: 1. B[1] = A[1] 2. for i = 2 to n do if (B[i 1] > 0) then B[i] = B[i 1] + A[i] else B[i] = A[i] 3. Find and print the maximum value in B[1.. n] Running time of Algorithm III: Step 1: O(1) time. Step 2: The loop executes at most n times and each iteration of the loop uses O(1) time. So, the total time for the loop is O(n). 25

26 Step 3: O(n) time. So, the overall running time is O(n). Conclusion: Algorithm III has the best running time for the Maximum Block Sum problem. Definition: An algorithm is efficient if its running time is a polynomial function of the input size. Note: Polynomial means that the running time can be expressed as O(n k ) where n is the problem size and k is a constant independent of n. 26

27 Examples: The algorithm for finding the maximum runs in O(n) time. The algorithm for the Zero Sum problem runs in O(n 2 ) time. All the three algorithms for the Maximum Block Sum problem are efficient algorithms. (Their respective running times are O(n 3 ), O(n 2 ) and O(n).) Problems such as Sorting and Shortest Path can also be solved efficiently. Exercise: Several algorithms with running times of O(n log 2 n) are known for the sorting problem. Also, given a sorted array A[1.. n] and a value q, binary search can be used to determine whether or not A contains the value q in O(log 2 n) time. Use these facts to devise an O(n log 2 n) algorithm for the Zero Sum problem. 27

28 NP-Complete Problems: Terminology: The class P contains all the problems for which a solution can be found in polynomial time. The class NP contains all the problems for which a given solution can be verified in polynomial time. Notes: 1. NP denotes Nondeterministic Polynomial. 2. For mathematical convenience, the class NP is restricted to decision problems. 28

29 Example: SAT is in NP. Given an assignment of values to the Boolean variables of a formula F, we can evaluate F and thus determine whether the given assignment satisfies the formula in polynomial time. However, we don t know how to find a satisfying assignment in polynomial time. Note: Every problem in P is also in NP. (Thus, P NP.) Other Problems in NP: Problem 7: Clique Input: A set S of n people, a set P of pairs of people from S who know each other and an integer K n. Question: Is there a subset S of S containing at least K people such that each person in S knows every other person in S? 29

30 Problem 8: Subset Sum Input: A set S of n integers and another integer Q. Question: Is there a subset S of S such that the sum of the integers in S is equal to Q? Problem 9: Longest Path Input: A graph G with vertex set V, edge set E, two vertices u and v and an integer K V. Question: Is there a path of length at least K between u and v? NP-Complete Problems: The hardest problems in NP. 30

31 These problems are equivalent in the following sense: (a) If any one of the NP-complete problems can be solved in polynomial time, then all of them can be solved in polynomial time. (If this happens, then P = NP.) (b) If we can show that there is no polynomial algorithm for any one of the NP-complete problems, then none of them can be solved in polynomial time. (If this happens, then P NP.) The problems SAT, Clique, Subset Sum and Longest Path defined above are known to be NP-complete. Thousands of problems that arise in practical applications are known to be NP-complete. P? = NP is a major open question. 31

Lecture 7: NP-Complete Problems

Lecture 7: NP-Complete Problems IAS/PCMI Summer Session 2000 Clay Mathematics Undergraduate Program Basic Course on Computational Complexity Lecture 7: NP-Complete Problems David Mix Barrington and Alexis Maciel July 25, 2000 1. Circuit

More information

NP-Completeness. CptS 223 Advanced Data Structures. Larry Holder School of Electrical Engineering and Computer Science Washington State University

NP-Completeness. CptS 223 Advanced Data Structures. Larry Holder School of Electrical Engineering and Computer Science Washington State University NP-Completeness CptS 223 Advanced Data Structures Larry Holder School of Electrical Engineering and Computer Science Washington State University 1 Hard Graph Problems Hard means no known solutions with

More information

Discrete Mathematics, Chapter 3: Algorithms

Discrete Mathematics, Chapter 3: Algorithms Discrete Mathematics, Chapter 3: Algorithms Richard Mayr University of Edinburgh, UK Richard Mayr (University of Edinburgh, UK) Discrete Mathematics. Chapter 3 1 / 28 Outline 1 Properties of Algorithms

More information

Discuss the size of the instance for the minimum spanning tree problem.

Discuss the size of the instance for the minimum spanning tree problem. 3.1 Algorithm complexity The algorithms A, B are given. The former has complexity O(n 2 ), the latter O(2 n ), where n is the size of the instance. Let n A 0 be the size of the largest instance that can

More information

Algorithms 2/17/2015. Double Summations. Enough Mathematical Appetizers! Algorithms. Algorithms. Algorithm Examples. Algorithm Examples

Algorithms 2/17/2015. Double Summations. Enough Mathematical Appetizers! Algorithms. Algorithms. Algorithm Examples. Algorithm Examples Double Summations Table 2 in 4 th Edition: Section 1.7 5 th Edition: Section.2 6 th and 7 th Edition: Section 2.4 contains some very useful formulas for calculating sums. Enough Mathematical Appetizers!

More information

Introduction to Algorithms. Part 3: P, NP Hard Problems

Introduction to Algorithms. Part 3: P, NP Hard Problems Introduction to Algorithms Part 3: P, NP Hard Problems 1) Polynomial Time: P and NP 2) NP-Completeness 3) Dealing with Hard Problems 4) Lower Bounds 5) Books c Wayne Goddard, Clemson University, 2004 Chapter

More information

Page 1. CSCE 310J Data Structures & Algorithms. CSCE 310J Data Structures & Algorithms. P, NP, and NP-Complete. Polynomial-Time Algorithms

Page 1. CSCE 310J Data Structures & Algorithms. CSCE 310J Data Structures & Algorithms. P, NP, and NP-Complete. Polynomial-Time Algorithms CSCE 310J Data Structures & Algorithms P, NP, and NP-Complete Dr. Steve Goddard goddard@cse.unl.edu CSCE 310J Data Structures & Algorithms Giving credit where credit is due:» Most of the lecture notes

More information

CMPS 102 Solutions to Homework 1

CMPS 102 Solutions to Homework 1 CMPS 0 Solutions to Homework Lindsay Brown, lbrown@soe.ucsc.edu September 9, 005 Problem..- p. 3 For inputs of size n insertion sort runs in 8n steps, while merge sort runs in 64n lg n steps. For which

More information

Outline. NP-completeness. When is a problem easy? When is a problem hard? Today. Euler Circuits

Outline. NP-completeness. When is a problem easy? When is a problem hard? Today. Euler Circuits Outline NP-completeness Examples of Easy vs. Hard problems Euler circuit vs. Hamiltonian circuit Shortest Path vs. Longest Path 2-pairs sum vs. general Subset Sum Reducing one problem to another Clique

More information

1. Nondeterministically guess a solution (called a certificate) 2. Check whether the solution solves the problem (called verification)

1. Nondeterministically guess a solution (called a certificate) 2. Check whether the solution solves the problem (called verification) Some N P problems Computer scientists have studied many N P problems, that is, problems that can be solved nondeterministically in polynomial time. Traditionally complexity question are studied as languages:

More information

Data Structures and Algorithms Written Examination

Data Structures and Algorithms Written Examination Data Structures and Algorithms Written Examination 22 February 2013 FIRST NAME STUDENT NUMBER LAST NAME SIGNATURE Instructions for students: Write First Name, Last Name, Student Number and Signature where

More information

Reductions & NP-completeness as part of Foundations of Computer Science undergraduate course

Reductions & NP-completeness as part of Foundations of Computer Science undergraduate course Reductions & NP-completeness as part of Foundations of Computer Science undergraduate course Alex Angelopoulos, NTUA January 22, 2015 Outline Alex Angelopoulos (NTUA) FoCS: Reductions & NP-completeness-

More information

2.3 Scheduling jobs on identical parallel machines

2.3 Scheduling jobs on identical parallel machines 2.3 Scheduling jobs on identical parallel machines There are jobs to be processed, and there are identical machines (running in parallel) to which each job may be assigned Each job = 1,,, must be processed

More information

Analysis of Binary Search algorithm and Selection Sort algorithm

Analysis of Binary Search algorithm and Selection Sort algorithm Analysis of Binary Search algorithm and Selection Sort algorithm In this section we shall take up two representative problems in computer science, work out the algorithms based on the best strategy to

More information

2. (a) Explain the strassen s matrix multiplication. (b) Write deletion algorithm, of Binary search tree. [8+8]

2. (a) Explain the strassen s matrix multiplication. (b) Write deletion algorithm, of Binary search tree. [8+8] Code No: R05220502 Set No. 1 1. (a) Describe the performance analysis in detail. (b) Show that f 1 (n)+f 2 (n) = 0(max(g 1 (n), g 2 (n)) where f 1 (n) = 0(g 1 (n)) and f 2 (n) = 0(g 2 (n)). [8+8] 2. (a)

More information

Complexity Theory. IE 661: Scheduling Theory Fall 2003 Satyaki Ghosh Dastidar

Complexity Theory. IE 661: Scheduling Theory Fall 2003 Satyaki Ghosh Dastidar Complexity Theory IE 661: Scheduling Theory Fall 2003 Satyaki Ghosh Dastidar Outline Goals Computation of Problems Concepts and Definitions Complexity Classes and Problems Polynomial Time Reductions Examples

More information

Approximation Algorithms

Approximation Algorithms Approximation Algorithms or: How I Learned to Stop Worrying and Deal with NP-Completeness Ong Jit Sheng, Jonathan (A0073924B) March, 2012 Overview Key Results (I) General techniques: Greedy algorithms

More information

CS325: Analysis of Algorithms, Fall Midterm Solutions

CS325: Analysis of Algorithms, Fall Midterm Solutions CS325: Analysis of Algorithms, Fall 2016 Midterm Solutions I don t know policy: you may write I don t know and nothing else to answer a question and receive 25 percent of the total points for that problem

More information

Computer Algorithms. NP-Complete Problems. CISC 4080 Yanjun Li

Computer Algorithms. NP-Complete Problems. CISC 4080 Yanjun Li Computer Algorithms NP-Complete Problems NP-completeness The quest for efficient algorithms is about finding clever ways to bypass the process of exhaustive search, using clues from the input in order

More information

CSC 373: Algorithm Design and Analysis Lecture 16

CSC 373: Algorithm Design and Analysis Lecture 16 CSC 373: Algorithm Design and Analysis Lecture 16 Allan Borodin February 25, 2013 Some materials are from Stephen Cook s IIT talk and Keven Wayne s slides. 1 / 17 Announcements and Outline Announcements

More information

Efficiency of algorithms. Algorithms. Efficiency of algorithms. Binary search and linear search. Best, worst and average case.

Efficiency of algorithms. Algorithms. Efficiency of algorithms. Binary search and linear search. Best, worst and average case. Algorithms Efficiency of algorithms Computational resources: time and space Best, worst and average case performance How to compare algorithms: machine-independent measure of efficiency Growth rate Complexity

More information

Introduction to Logic in Computer Science: Autumn 2006

Introduction to Logic in Computer Science: Autumn 2006 Introduction to Logic in Computer Science: Autumn 2006 Ulle Endriss Institute for Logic, Language and Computation University of Amsterdam Ulle Endriss 1 Plan for Today Now that we have a basic understanding

More information

Theoretical Computer Science (Bridging Course) Complexity

Theoretical Computer Science (Bridging Course) Complexity Theoretical Computer Science (Bridging Course) Complexity Gian Diego Tipaldi A scenario You are a programmer working for a logistics company Your boss asks you to implement a program that optimizes the

More information

Minimum Spanning Trees

Minimum Spanning Trees Minimum Spanning Trees Algorithms and 18.304 Presentation Outline 1 Graph Terminology Minimum Spanning Trees 2 3 Outline Graph Terminology Minimum Spanning Trees 1 Graph Terminology Minimum Spanning Trees

More information

CS 4310 HOMEWORK SET 1

CS 4310 HOMEWORK SET 1 CS 4310 HOMEWORK SET 1 PAUL MILLER Section 2.1 Exercises Exercise 2.1-1. Using Figure 2.2 as a model, illustrate the operation of INSERTION-SORT on the array A = 31, 41, 59, 26, 41, 58. Solution: Assume

More information

Chapter. NP-Completeness. Contents

Chapter. NP-Completeness. Contents Chapter 13 NP-Completeness Contents 13.1 P and NP......................... 593 13.1.1 Defining the Complexity Classes P and NP...594 13.1.2 Some Interesting Problems in NP.......... 597 13.2 NP-Completeness....................

More information

In mathematics, it is often important to get a handle on the error term of an approximation. For instance, people will write

In mathematics, it is often important to get a handle on the error term of an approximation. For instance, people will write Big O notation (with a capital letter O, not a zero), also called Landau's symbol, is a symbolism used in complexity theory, computer science, and mathematics to describe the asymptotic behavior of functions.

More information

Chapter 11. 11.1 Load Balancing. Approximation Algorithms. Load Balancing. Load Balancing on 2 Machines. Load Balancing: Greedy Scheduling

Chapter 11. 11.1 Load Balancing. Approximation Algorithms. Load Balancing. Load Balancing on 2 Machines. Load Balancing: Greedy Scheduling Approximation Algorithms Chapter Approximation Algorithms Q. Suppose I need to solve an NP-hard problem. What should I do? A. Theory says you're unlikely to find a poly-time algorithm. Must sacrifice one

More information

Tutorial 8. NP-Complete Problems

Tutorial 8. NP-Complete Problems Tutorial 8 NP-Complete Problems Decision Problem Statement of a decision problem Part 1: instance description defining the input Part 2: question stating the actual yesor-no question A decision problem

More information

Lecture 2: The Complexity of Some Problems

Lecture 2: The Complexity of Some Problems IAS/PCMI Summer Session 2000 Clay Mathematics Undergraduate Program Basic Course on Computational Complexity Lecture 2: The Complexity of Some Problems David Mix Barrington and Alexis Maciel July 18, 2000

More information

Cost Model: Work, Span and Parallelism. 1 The RAM model for sequential computation:

Cost Model: Work, Span and Parallelism. 1 The RAM model for sequential computation: CSE341T 08/31/2015 Lecture 3 Cost Model: Work, Span and Parallelism In this lecture, we will look at how one analyze a parallel program written using Cilk Plus. When we analyze the cost of an algorithm

More information

Algorithms and Data Structures

Algorithms and Data Structures Algorithm Analysis Page 1 BFH-TI: Softwareschule Schweiz Algorithm Analysis Dr. CAS SD01 Algorithm Analysis Page 2 Outline Course and Textbook Overview Analysis of Algorithm Pseudo-Code and Primitive Operations

More information

SIMS 255 Foundations of Software Design. Complexity and NP-completeness

SIMS 255 Foundations of Software Design. Complexity and NP-completeness SIMS 255 Foundations of Software Design Complexity and NP-completeness Matt Welsh November 29, 2001 mdw@cs.berkeley.edu 1 Outline Complexity of algorithms Space and time complexity ``Big O'' notation Complexity

More information

Introduction to computer science

Introduction to computer science Introduction to computer science Michael A. Nielsen University of Queensland Goals: 1. Introduce the notion of the computational complexity of a problem, and define the major computational complexity classes.

More information

1. What s wrong with the following proofs by induction?

1. What s wrong with the following proofs by induction? ArsDigita University Month : Discrete Mathematics - Professor Shai Simonson Problem Set 4 Induction and Recurrence Equations Thanks to Jeffrey Radcliffe and Joe Rizzo for many of the solutions. Pasted

More information

The Classes P and NP

The Classes P and NP The Classes P and NP We now shift gears slightly and restrict our attention to the examination of two families of problems which are very important to computer scientists. These families constitute the

More information

Answers to Homework 1

Answers to Homework 1 Answers to Homework 1 p. 13 1.2 3 What is the smallest value of n such that an algorithm whose running time is 100n 2 runs faster than an algorithm whose running time is 2 n on the same machine? Find the

More information

Algorithms, Integers

Algorithms, Integers CHAPTER 3 Algorithms, Integers 3.1. Algorithms Consider the following list of instructions to find the maximum of three numbers a, b, c: 1. Assign variable x the value of a. 2. If b > x then assign x the

More information

! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. !-approximation algorithm.

! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. !-approximation algorithm. Approximation Algorithms Chapter Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of

More information

Near Optimal Solutions

Near Optimal Solutions Near Optimal Solutions Many important optimization problems are lacking efficient solutions. NP-Complete problems unlikely to have polynomial time solutions. Good heuristics important for such problems.

More information

Chapter 3. if 2 a i then location: = i. Page 40

Chapter 3. if 2 a i then location: = i. Page 40 Chapter 3 1. Describe an algorithm that takes a list of n integers a 1,a 2,,a n and finds the number of integers each greater than five in the list. Ans: procedure greaterthanfive(a 1,,a n : integers)

More information

Catalan Numbers. Thomas A. Dowling, Department of Mathematics, Ohio State Uni- versity.

Catalan Numbers. Thomas A. Dowling, Department of Mathematics, Ohio State Uni- versity. 7 Catalan Numbers Thomas A. Dowling, Department of Mathematics, Ohio State Uni- Author: versity. Prerequisites: The prerequisites for this chapter are recursive definitions, basic counting principles,

More information

Welcome to... Problem Analysis and Complexity Theory 716.054, 3 VU

Welcome to... Problem Analysis and Complexity Theory 716.054, 3 VU Welcome to... Problem Analysis and Complexity Theory 716.054, 3 VU Birgit Vogtenhuber Institute for Software Technology email: bvogt@ist.tugraz.at office hour: Tuesday 10:30 11:30 slides: http://www.ist.tugraz.at/pact.html

More information

P versus NP, and More

P versus NP, and More 1 P versus NP, and More Great Ideas in Theoretical Computer Science Saarland University, Summer 2014 If you have tried to solve a crossword puzzle, you know that it is much harder to solve it than to verify

More information

On the Relationship between Classes P and NP

On the Relationship between Classes P and NP Journal of Computer Science 8 (7): 1036-1040, 2012 ISSN 1549-3636 2012 Science Publications On the Relationship between Classes P and NP Anatoly D. Plotnikov Department of Computer Systems and Networks,

More information

MapReduce and Distributed Data Analysis. Sergei Vassilvitskii Google Research

MapReduce and Distributed Data Analysis. Sergei Vassilvitskii Google Research MapReduce and Distributed Data Analysis Google Research 1 Dealing With Massive Data 2 2 Dealing With Massive Data Polynomial Memory Sublinear RAM Sketches External Memory Property Testing 3 3 Dealing With

More information

AS-2261 M.Sc.(First Semester) Examination-2013 Paper -fourth Subject-Data structure with algorithm

AS-2261 M.Sc.(First Semester) Examination-2013 Paper -fourth Subject-Data structure with algorithm AS-2261 M.Sc.(First Semester) Examination-2013 Paper -fourth Subject-Data structure with algorithm Time: Three Hours] [Maximum Marks: 60 Note Attempts all the questions. All carry equal marks Section A

More information

Algorithm Design and Analysis Homework #6 Due: 1pm, Monday, January 9, === Homework submission instructions ===

Algorithm Design and Analysis Homework #6 Due: 1pm, Monday, January 9, === Homework submission instructions === Algorithm Design and Analysis Homework #6 Due: 1pm, Monday, January 9, 2012 === Homework submission instructions === Submit the answers for writing problems (including your programming report) through

More information

Introduction to Algorithms Review information for Prelim 1 CS 4820, Spring 2010 Distributed Wednesday, February 24

Introduction to Algorithms Review information for Prelim 1 CS 4820, Spring 2010 Distributed Wednesday, February 24 Introduction to Algorithms Review information for Prelim 1 CS 4820, Spring 2010 Distributed Wednesday, February 24 The final exam will cover seven topics. 1. greedy algorithms 2. divide-and-conquer algorithms

More information

princeton univ. F 13 cos 521: Advanced Algorithm Design Lecture 6: Provable Approximation via Linear Programming Lecturer: Sanjeev Arora

princeton univ. F 13 cos 521: Advanced Algorithm Design Lecture 6: Provable Approximation via Linear Programming Lecturer: Sanjeev Arora princeton univ. F 13 cos 521: Advanced Algorithm Design Lecture 6: Provable Approximation via Linear Programming Lecturer: Sanjeev Arora Scribe: One of the running themes in this course is the notion of

More information

The Knapsack Problem. Decisions and State. Dynamic Programming Solution Introduction to Algorithms Recitation 19 November 23, 2011

The Knapsack Problem. Decisions and State. Dynamic Programming Solution Introduction to Algorithms Recitation 19 November 23, 2011 The Knapsack Problem You find yourself in a vault chock full of valuable items. However, you only brought a knapsack of capacity S pounds, which means the knapsack will break down if you try to carry more

More information

Algorithm Design and Analysis

Algorithm Design and Analysis Algorithm Design and Analysis LECTURE 27 Approximation Algorithms Load Balancing Weighted Vertex Cover Reminder: Fill out SRTEs online Don t forget to click submit Sofya Raskhodnikova 12/6/2011 S. Raskhodnikova;

More information

NP-complete? NP-hard? Some Foundations of Complexity. Prof. Sven Hartmann Clausthal University of Technology Department of Informatics

NP-complete? NP-hard? Some Foundations of Complexity. Prof. Sven Hartmann Clausthal University of Technology Department of Informatics NP-complete? NP-hard? Some Foundations of Complexity Prof. Sven Hartmann Clausthal University of Technology Department of Informatics Tractability of Problems Some problems are undecidable: no computer

More information

Scheduling. Open shop, job shop, flow shop scheduling. Related problems. Open shop, job shop, flow shop scheduling

Scheduling. Open shop, job shop, flow shop scheduling. Related problems. Open shop, job shop, flow shop scheduling Scheduling Basic scheduling problems: open shop, job shop, flow job The disjunctive graph representation Algorithms for solving the job shop problem Computational complexity of the job shop problem Open

More information

Warshall s Algorithm: Transitive Closure

Warshall s Algorithm: Transitive Closure CS 0 Theory of Algorithms / CS 68 Algorithms in Bioinformaticsi Dynamic Programming Part II. Warshall s Algorithm: Transitive Closure Computes the transitive closure of a relation (Alternatively: all paths

More information

A simple algorithm with no simple verication

A simple algorithm with no simple verication A simple algorithm with no simple verication Laszlo Csirmaz Central European University Abstract The correctness of a simple sorting algorithm is resented, which algorithm is \evidently wrong" at the rst

More information

Solutions to Homework 6

Solutions to Homework 6 Solutions to Homework 6 Debasish Das EECS Department, Northwestern University ddas@northwestern.edu 1 Problem 5.24 We want to find light spanning trees with certain special properties. Given is one example

More information

Background. Linear Convolutional Encoders (2) Output as Function of Input (1) Linear Convolutional Encoders (1)

Background. Linear Convolutional Encoders (2) Output as Function of Input (1) Linear Convolutional Encoders (1) S-723410 Convolutional Codes (1) 1 S-723410 Convolutional Codes (1) 3 Background Convolutional codes do not segment the data stream, but convert the entire data stream into another stream (codeword) Some

More information

Answers to some of the exercises.

Answers to some of the exercises. Answers to some of the exercises. Chapter 2. Ex.2.1 (a) There are several ways to do this. Here is one possibility. The idea is to apply the k-center algorithm first to D and then for each center in D

More information

SYSM 6304: Risk and Decision Analysis Lecture 5: Methods of Risk Analysis

SYSM 6304: Risk and Decision Analysis Lecture 5: Methods of Risk Analysis SYSM 6304: Risk and Decision Analysis Lecture 5: Methods of Risk Analysis M. Vidyasagar Cecil & Ida Green Chair The University of Texas at Dallas Email: M.Vidyasagar@utdallas.edu October 17, 2015 Outline

More information

Relations Graphical View

Relations Graphical View Relations Slides by Christopher M. Bourke Instructor: Berthe Y. Choueiry Introduction Recall that a relation between elements of two sets is a subset of their Cartesian product (of ordered pairs). A binary

More information

Graph Algorithms using Map-Reduce

Graph Algorithms using Map-Reduce Graph Algorithms using Map-Reduce Graphs are ubiquitous in modern society. Some examples: The hyperlink structure of the web 1/7 Graph Algorithms using Map-Reduce Graphs are ubiquitous in modern society.

More information

CSE373: Data Structures and Algorithms Lecture 3: Math Review; Algorithm Analysis. Linda Shapiro Winter 2015

CSE373: Data Structures and Algorithms Lecture 3: Math Review; Algorithm Analysis. Linda Shapiro Winter 2015 CSE373: Data Structures and Algorithms Lecture 3: Math Review; Algorithm Analysis Linda Shapiro Today Registration should be done. Homework 1 due 11:59 pm next Wednesday, January 14 Review math essential

More information

CS/COE 1501 http://cs.pitt.edu/~bill/1501/

CS/COE 1501 http://cs.pitt.edu/~bill/1501/ CS/COE 1501 http://cs.pitt.edu/~bill/1501/ Lecture 01 Course Introduction Meta-notes These notes are intended for use by students in CS1501 at the University of Pittsburgh. They are provided free of charge

More information

Analysis of Computer Algorithms. Algorithm. Algorithm, Data Structure, Program

Analysis of Computer Algorithms. Algorithm. Algorithm, Data Structure, Program Analysis of Computer Algorithms Hiroaki Kobayashi Input Algorithm Output 12/13/02 Algorithm Theory 1 Algorithm, Data Structure, Program Algorithm Well-defined, a finite step-by-step computational procedure

More information

Binary Heaps * * * * * * * / / \ / \ / \ / \ / \ * * * * * * * * * * * / / \ / \ / / \ / \ * * * * * * * * * *

Binary Heaps * * * * * * * / / \ / \ / \ / \ / \ * * * * * * * * * * * / / \ / \ / / \ / \ * * * * * * * * * * Binary Heaps A binary heap is another data structure. It implements a priority queue. Priority Queue has the following operations: isempty add (with priority) remove (highest priority) peek (at highest

More information

Why Study NP- hardness. NP Hardness/Completeness Overview. P and NP. Scaling 9/3/13. Ron Parr CPS 570. NP hardness is not an AI topic

Why Study NP- hardness. NP Hardness/Completeness Overview. P and NP. Scaling 9/3/13. Ron Parr CPS 570. NP hardness is not an AI topic Why Study NP- hardness NP Hardness/Completeness Overview Ron Parr CPS 570 NP hardness is not an AI topic It s important for all computer scienhsts Understanding it will deepen your understanding of AI

More information

Applied Algorithm Design Lecture 5

Applied Algorithm Design Lecture 5 Applied Algorithm Design Lecture 5 Pietro Michiardi Eurecom Pietro Michiardi (Eurecom) Applied Algorithm Design Lecture 5 1 / 86 Approximation Algorithms Pietro Michiardi (Eurecom) Applied Algorithm Design

More information

Approximation Algorithms: LP Relaxation, Rounding, and Randomized Rounding Techniques. My T. Thai

Approximation Algorithms: LP Relaxation, Rounding, and Randomized Rounding Techniques. My T. Thai Approximation Algorithms: LP Relaxation, Rounding, and Randomized Rounding Techniques My T. Thai 1 Overview An overview of LP relaxation and rounding method is as follows: 1. Formulate an optimization

More information

Single machine parallel batch scheduling with unbounded capacity

Single machine parallel batch scheduling with unbounded capacity Workshop on Combinatorics and Graph Theory 21th, April, 2006 Nankai University Single machine parallel batch scheduling with unbounded capacity Yuan Jinjiang Department of mathematics, Zhengzhou University

More information

OHJ-2306 Introduction to Theoretical Computer Science, Fall 2012 8.11.2012

OHJ-2306 Introduction to Theoretical Computer Science, Fall 2012 8.11.2012 276 The P vs. NP problem is a major unsolved problem in computer science It is one of the seven Millennium Prize Problems selected by the Clay Mathematics Institute to carry a $ 1,000,000 prize for the

More information

Section IV.1: Recursive Algorithms and Recursion Trees

Section IV.1: Recursive Algorithms and Recursion Trees Section IV.1: Recursive Algorithms and Recursion Trees Definition IV.1.1: A recursive algorithm is an algorithm that solves a problem by (1) reducing it to an instance of the same problem with smaller

More information

! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. #-approximation algorithm.

! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. #-approximation algorithm. Approximation Algorithms 11 Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of three

More information

NP-Completeness and Cook s Theorem

NP-Completeness and Cook s Theorem NP-Completeness and Cook s Theorem Lecture notes for COM3412 Logic and Computation 15th January 2002 1 NP decision problems The decision problem D L for a formal language L Σ is the computational task:

More information

Solutions to Homework 6

Solutions to Homework 6 Solutions to Homework 6 Section 3.1 4. Set the answer to be. For i going from 1 through n 1, compute the value of the (i + 1)st element in the list minus the ist element in the list. If this is larger

More information

CAD Algorithms. P and NP

CAD Algorithms. P and NP CAD Algorithms The Classes P and NP Mohammad Tehranipoor ECE Department 6 September 2010 1 P and NP P and NP are two families of problems. P is a class which contains all of the problems we solve using

More information

The divide and conquer strategy has three basic parts. For a given problem of size n,

The divide and conquer strategy has three basic parts. For a given problem of size n, 1 Divide & Conquer One strategy for designing efficient algorithms is the divide and conquer approach, which is also called, more simply, a recursive approach. The analysis of recursive algorithms often

More information

Data Structures. Algorithm Performance and Big O Analysis

Data Structures. Algorithm Performance and Big O Analysis Data Structures Algorithm Performance and Big O Analysis What s an Algorithm? a clearly specified set of instructions to be followed to solve a problem. In essence: A computer program. In detail: Defined

More information

Logic in Computer Science: Logic Gates

Logic in Computer Science: Logic Gates Logic in Computer Science: Logic Gates Lila Kari The University of Western Ontario Logic in Computer Science: Logic Gates CS2209, Applied Logic for Computer Science 1 / 49 Logic and bit operations Computers

More information

Determinants can be used to solve a linear system of equations using Cramer s Rule.

Determinants can be used to solve a linear system of equations using Cramer s Rule. 2.6.2 Cramer s Rule Determinants can be used to solve a linear system of equations using Cramer s Rule. Cramer s Rule for Two Equations in Two Variables Given the system This system has the unique solution

More information

One last point: we started off this book by introducing another famously hard search problem:

One last point: we started off this book by introducing another famously hard search problem: S. Dasgupta, C.H. Papadimitriou, and U.V. Vazirani 261 Factoring One last point: we started off this book by introducing another famously hard search problem: FACTORING, the task of finding all prime factors

More information

Bicolored Shortest Paths in Graphs with Applications to Network Overlay Design

Bicolored Shortest Paths in Graphs with Applications to Network Overlay Design Bicolored Shortest Paths in Graphs with Applications to Network Overlay Design Hongsik Choi and Hyeong-Ah Choi Department of Electrical Engineering and Computer Science George Washington University Washington,

More information

Introduction to NP-Completeness Written and copyright c by Jie Wang 1

Introduction to NP-Completeness Written and copyright c by Jie Wang 1 91.502 Foundations of Comuter Science 1 Introduction to Written and coyright c by Jie Wang 1 We use time-bounded (deterministic and nondeterministic) Turing machines to study comutational comlexity of

More information

The Traveling Beams Optical Solutions for Bounded NP-Complete Problems

The Traveling Beams Optical Solutions for Bounded NP-Complete Problems The Traveling Beams Optical Solutions for Bounded NP-Complete Problems Shlomi Dolev, Hen Fitoussi Abstract Architectures for optical processors designed to solve bounded instances of NP-Complete problems

More information

Steven Skiena. skiena

Steven Skiena.  skiena Lecture 3: Recurrence Relations (1997) Steven Skiena Department of Computer Science State University of New York Stony Brook, NY 11794 4400 http://www.cs.sunysb.edu/ skiena Argue the solution to T(n) =

More information

Solution of Linear Systems

Solution of Linear Systems Chapter 3 Solution of Linear Systems In this chapter we study algorithms for possibly the most commonly occurring problem in scientific computing, the solution of linear systems of equations. We start

More information

CSC 180 H1F Algorithm Runtime Analysis Lecture Notes Fall 2015

CSC 180 H1F Algorithm Runtime Analysis Lecture Notes Fall 2015 1 Introduction These notes introduce basic runtime analysis of algorithms. We would like to be able to tell if a given algorithm is time-efficient, and to be able to compare different algorithms. 2 Linear

More information

Mathematical Induction. Lecture 10-11

Mathematical Induction. Lecture 10-11 Mathematical Induction Lecture 10-11 Menu Mathematical Induction Strong Induction Recursive Definitions Structural Induction Climbing an Infinite Ladder Suppose we have an infinite ladder: 1. We can reach

More information

Mathematics for Algorithm and System Analysis

Mathematics for Algorithm and System Analysis Mathematics for Algorithm and System Analysis for students of computer and computational science Edward A. Bender S. Gill Williamson c Edward A. Bender & S. Gill Williamson 2005. All rights reserved. Preface

More information

Reminder: Complexity (1) Parallel Complexity Theory. Reminder: Complexity (2) Complexity-new

Reminder: Complexity (1) Parallel Complexity Theory. Reminder: Complexity (2) Complexity-new Reminder: Complexity (1) Parallel Complexity Theory Lecture 6 Number of steps or memory units required to compute some result In terms of input size Using a single processor O(1) says that regardless of

More information

Reminder: Complexity (1) Parallel Complexity Theory. Reminder: Complexity (2) Complexity-new GAP (2) Graph Accessibility Problem (GAP) (1)

Reminder: Complexity (1) Parallel Complexity Theory. Reminder: Complexity (2) Complexity-new GAP (2) Graph Accessibility Problem (GAP) (1) Reminder: Complexity (1) Parallel Complexity Theory Lecture 6 Number of steps or memory units required to compute some result In terms of input size Using a single processor O(1) says that regardless of

More information

Problem Set 7 Solutions

Problem Set 7 Solutions 8 8 Introduction to Algorithms May 7, 2004 Massachusetts Institute of Technology 6.046J/18.410J Professors Erik Demaine and Shafi Goldwasser Handout 25 Problem Set 7 Solutions This problem set is due in

More information

3.1 Algorithms. Pseudocode. Searching Algorithms.

3.1 Algorithms. Pseudocode. Searching Algorithms. 3.1 Algorithms. Pseudocode. Searching Algorithms. Outline: What is an Algorithm? Pseudocode Search Problems Sorting Problems (next) 1 3.1 Algorithms. Pseudocode. Searching Algorithms. Examples of Mathematical

More information

NP-completeness and the real world. NP completeness. NP-completeness and the real world (2) NP-completeness and the real world

NP-completeness and the real world. NP completeness. NP-completeness and the real world (2) NP-completeness and the real world -completeness and the real world completeness Course Discrete Biological Models (Modelli Biologici Discreti) Zsuzsanna Lipták Imagine you are working for a biotech company. One day your boss calls you

More information

ARTICLE IN PRESS. European Journal of Operational Research xxx (2004) xxx xxx. Discrete Optimization. Nan Kong, Andrew J.

ARTICLE IN PRESS. European Journal of Operational Research xxx (2004) xxx xxx. Discrete Optimization. Nan Kong, Andrew J. A factor 1 European Journal of Operational Research xxx (00) xxx xxx Discrete Optimization approximation algorithm for two-stage stochastic matching problems Nan Kong, Andrew J. Schaefer * Department of

More information

How to prove an algorithm is correct?

How to prove an algorithm is correct? How to prove an algorithm is correct? To prove the incorrectness of an algorithm, one counter-example is enough. Proving the correctness of an algorithm is similar to proving a mathematical theorem; fundamentally,

More information

Classical Information and Bits

Classical Information and Bits p. 1/24 Classical Information and Bits The simplest variable that can carry information is the bit, which can take only two values, 0 or 1. Any ensemble, description, or set of discrete values can be quantified

More information

Sample Problems in Discrete Mathematics

Sample Problems in Discrete Mathematics Sample Problems in Discrete Mathematics This handout lists some sample problems that you should be able to solve as a pre-requisite to Computer Algorithms Try to solve all of them You should also read

More information

Memoization/Dynamic Programming. The String reconstruction problem. CS125 Lecture 5 Fall 2016

Memoization/Dynamic Programming. The String reconstruction problem. CS125 Lecture 5 Fall 2016 CS125 Lecture 5 Fall 2016 Memoization/Dynamic Programming Today s lecture discusses memoization, which is a method for speeding up algorithms based on recursion, by using additional memory to remember

More information

DATA ANALYSIS II. Matrix Algorithms

DATA ANALYSIS II. Matrix Algorithms DATA ANALYSIS II Matrix Algorithms Similarity Matrix Given a dataset D = {x i }, i=1,..,n consisting of n points in R d, let A denote the n n symmetric similarity matrix between the points, given as where

More information