Analysis of Computer Algorithms. Algorithm. Algorithm, Data Structure, Program

Save this PDF as:

Size: px
Start display at page:

Transcription

1 Analysis of Computer Algorithms Hiroaki Kobayashi Input Algorithm Output 12/13/02 Algorithm Theory 1 Algorithm, Data Structure, Program Algorithm Well-defined, a finite step-by-step computational procedure that takes some values, or set of values, as input and produces some value, or set of values, as output. Procedure a sequence of computational steps that transform the input into the output. Finiteness of the steps of a procedure is not guaranteed. Program algorithm represented in a program language Data structure data structure suited for input, output, and temporal data during computing. Passive objects Program = Algorithm + Data Structure 12/13/02 Algorithm Theory 2

2 What is a Computable Problem Computable (Solvable) problem There is an algorithm with finite program steps to solve the problem on a computer Computable Problems incomputable Many different algorithms for a computable problem are available! Analysis of algorithms is essential to design good algorithms 12/13/02 Algorithm Theory 3 How can we evaluate algorithms? Measure running time of a program Most algorithms transform input objects into output objects. The running time of an algorithm typically grows with the input size. Average case time is often difficult to determine. We focus on the worst case running time. Easier to analyze Crucial to applications such as games, finance and robotics Running Time best case average case worst case Input Size 12/13/02 Algorithm Theory 4

3 Experimental Studies Write a program implementing the algorithm Run the program with inputs of varying size and composition Use a method like System.currentTimeMillis() to get an accurate measure of the actual running time Plot the results Time (ms) Input Size 12/13/02 Algorithm Theory 5 Limitations of Experiments It is necessary to implement the algorithm, which may be difficult Results may not be indicative of the running time on other inputs not included in the experiment. In order to compare two algorithms, the same hardware and software environments must be used 12/13/02 Algorithm Theory 6

4 Theoretical Analysis Uses a high-level description of the algorithm instead of an implementation Characterizes running time as a function of the input size, n. Takes into account all possible inputs Allows us to evaluate the speed of an algorithm independent of the hardware/software environment 12/13/02 Algorithm Theory 7 Theoretical Analysis (Cont d) Algorithm analysis needs an abstract machine as a measure for evaluation of algorithms Abstract Machines: Very simple computer models for measurement of program execution steps and memory requirement Turing Machine Random Access Machine (RAM) Program correctness Computation complexity evaluation in terms of speed and memory 12/13/02 Algorithm Theory 8

5 Abstract Machine Example : Turing Machine Proposed by Alan M. Turing in 1936 (Pre-Electric-Computer Era) Processor Finite State Machine Tape Memory with unlimited cells Blank or symbol Head Head for tape reads and writes Turing Machine can perform any computation that any general purpose computer can, but has a quite different architecture from conventional computers. Processor (Finite State Machine) Head Cell Unlimited Tape (memory) 12/13/02 Algorithm Theory 9 Pseudo-code High-level description of an algorithm More structured than English prose Less detailed than a program Preferred notation for describing algorithms Hides program design issues Example: Find max element of an array Algorithm arraymax(a, n) Input array A of n integers Output maximum element of A currentmax A[0] for i 1 to n 1 do if A[i] > currentmax then currentmax A[i] return currentmax 12/13/02 Algorithm Theory 10

6 Pseudocode Details Control flow if then [else ] while do repeat until for do Indentation replaces braces Method declaration Algorithm method (arg [, arg ]) Input Output Method call var.method (arg [, arg ]) Return value return expression Expressions Assignment (like = in Java) = Equality testing (like == in Java) n 2 Superscripts and other mathematical formatting allowed 12/13/02 Algorithm Theory 11 Von-Neumann Computer Model: Base Model for Modern Computer Systems Computer System Processor Control Input Datapath (ALU, Reg) Memory Output 12/13/02 Algorithm Theory 12

7 The Random Access Machine (RAM) Model A CPU An potentially unbounded bank of memory cells, each of which can hold an arbitrary number or character 0 12 Memory cells are numbered and accessing any cell in memory takes unit time. 12/13/02 Algorithm Theory 13 Primitive Operations Basic computations performed by an algorithm Identifiable in pseudocode Largely independent from the programming language Exact definition not important Assumed to take a constant amount of time in the RAM model Examples: Evaluating an expression Assigning a value to a variable Indexing into an array Calling a method Returning from a method 12/13/02 Algorithm Theory 14

8 Analysis of Algorithm Complexity Execution Steps Number of primitive steps/operations to solve a given problem. Memory Requirement Total amount of memory required to hold inputs, outputs and temporal results Example: Search a desired datum among N data Case 1 A data set of N elements is randomly stored. Case 2 A data set of N elements is stored in an ascending/descending order. 12/13/02 Algorithm Theory 15 Search Algorithms Case 1 Random stored data Sequential search Case 2 Sorted data Binary search < log If smaller If larger /13/02 Algorithm Theory 16

9 Summary of Search Algorithms Sequential search Binary search Search Best 1, Worst N, Average N/2 searches log N Insert new data 1 log N Delete data N/2 searches for search, N/2 moves log N searches, N/2 moves Initial data arrangement for N times N N logn 12/13/02 Algorithm Theory 17 N vs. Log N :As a Function of Input Data Very slow growth Algorithms with log N steps are very good ones! 12/13/02 Algorithm Theory 18

10 Metrics for Complexity Analysis Many programs are extremely sensitive to their input data Algorithms are evaluated as a function of the number of input data. Two Metrics for Algorithm Analysis Space Complexity Total amount of memory to store inputs, outputs, and temporal data Time Complexity Number of (program) steps to solve the problem So, What s a Bad Algorithm, yes, it is computable, but it needs the enormous amount of memory the enormous number of execution steps 12/13/02 Algorithm Theory 19 Complexity in Worst Case and Average Case Worst Case Complexity The worst-case execution complexity of an algorithm gives an upper bound on the execution steps for any input. Evaluate the algorithm using the worst possible input configuration. Guarantee that the algorithm will never take any longer. For some algorithms, the worst case rarely occurs. In such a case, the worst case complexity does not reflect the actual program behavior. Average Complexity (Expected Complexity) Estimate the complexity of an algorithm with considering input occurrence probability. Evaluate the algorithm using typical input data. Reflects the actual behavior of an algorithm, but hard to constitute average inputs for a particular problem. Assume that all inputs of a given size are equally likely 12/13/02 Algorithm Theory 20

11 Estimating Running Time Assume that an algorithm executes 7n-1 primitive operations in the worst case. Define: a = Time taken by the fastest primitive operation b = Time taken by the slowest primitive operation Let T(n) be worst-case time of the algorithm. Then a(7n-1) T(n) b(7n-1) Hence, the running time T(n) is bounded by two linear functions. 12/13/02 Algorithm Theory 21 Growth Rate of Running Time Changing the hardware/ software environment Affects T(n) by a constant factor, but Does not alter the growth rate of T(n) The linear growth rate of the running time T(n) is an intrinsic property of the algorithm 12/13/02 Algorithm Theory 22

12 Growth Rates Growth rates of functions: Linear n Quadratic n 2 Cubic n 3 In a log-log chart, the slope of the line corresponds to the growth rate of the function T (n ) 1E+30 1E+28 1E+26 1E+24 1E+22 1E+20 1E+18 1E+16 1E+14 1E+12 1E+10 1E+8 1E+6 1E+4 1E+2 1E+0 Cubic Quadratic Linear 1E+0 1E+2 1E+4 1E+6 1E+8 1E+10 n 12/13/02 Algorithm Theory 23 Constant Factors The growth rate is not affected by constant factors or lower-order terms Examples 10 2 n is a linear function 10 5 n n is a quadratic function T (n ) 1E+26 1E+24 1E+22 1E+20 1E+18 1E+16 1E+14 1E+12 1E+10 1E+8 1E+6 1E+4 1E+2 1E+0 Quadratic Quadratic Linear Linear 1E+0 1E+2 1E+4 1E+6 1E+8 1E+10 n 12/13/02 Algorithm Theory 24

13 Big O-notation: Quantitative and Approximate Analysis of Algorithm Complexity Big O-notation gives an upper bound for a function to within a constant factor. If there exist positive constant n 0 and c such that to the right of n 0, the value of f(n) always lies on or below cg(n), i.e., f(n) cg(n) n: number of inputs The complexity of a given algorithm : O(g(n)) 12/13/02 Algorithm Theory 25 Analysis of Algorithm Complexity using Big O-Notation Example: An algorithm with 2n 2 execution steps is roughly evaluated with a complexity of O(n 2 ) steps O-notation gives approximate evaluation of algorithms ignore details!! Just consider the leading/dominant term of mathematical expression of algorithm complexity n 2, 2.5n 2, 100n 2, n+2n 2 O(n 2 )!! Free the analysis from considering the details of particular machine characteristics: independent of both inputs and implementation details 12/13/02 Algorithm Theory 26

14 Big-O Example Example: 2n + 10 is O(n) 2n + 10 cn (c 2) n 10 n 10/(c 2) 10,000 1, n 2n+10 n 10 Pick c = 3 and n 0 = ,000 n 12/13/02 Algorithm Theory 27 Big-O Example Example: function n 2 is not O(n) n 2 cn n c 1,000, ,000 10,000 1,000 n^2 100n 10n n The above inequality cannot be satisfied since c must be a constant ,000 n 12/13/02 Algorithm Theory 28

15 More Big-O Examples 7n-2 is O(n) need c > 0 and n 0 1 such that 7n-2 c n for n n 0 this is true for c = 7 and n 0 = 1 3n n is O(n 3 ) need c > 0 and n 0 1 such that 3n n c n 3 for n n 0 this is true for c = 4 and n 0 = 21 3 log n + log log n is O(log n) need c > 0 and n 0 1 such that 3 log n + log log n c log n for n n 0 this is true for c = 4 and n 0 = 2 12/13/02 Algorithm Theory 29 Big-Oh and Growth Rate The big-oh notation gives an upper bound on the growth rate of a function The statement f(n) is O(g(n)) means that the growth rate of f(n) is no more than the growth rate of g(n) We can use the big-oh notation to rank functions according to their growth rate f(n) is O(g(n)) g(n) is O(f(n)) g(n) grows more f(n) grows more Same growth Yes No Yes No Yes Yes 12/13/02 Algorithm Theory 30

16 Big-Oh Rules If is f(n) a polynomial of degree d, then f(n) is O(n d ), i.e., 1. Drop lower-order terms 2. Drop constant factors Use the smallest possible class of functions Say 2n is O(n) instead of 2n is O(n 2 ) Use the simplest expression of the class Say 3n + 5is O(n) instead of 3n + 5is O(3n) 12/13/02 Algorithm Theory 31 Asymptotic Algorithm Analysis The asymptotic analysis of an algorithm determines the running time in big-oh notation To perform the asymptotic analysis We find the worst-case number of primitive operations executed as a function of the input size We express this function with big-oh notation Example: We determine that the algorithm executes at most 7n 1 primitive operations We say that the algorithm runs in O(n) time Since constant factors and lower-order terms are eventually dropped anyhow, we can disregard them when counting primitive operations 12/13/02 Algorithm Theory 32

17 Good Algorithms Good Algorithm or Bad Algorithm?: Quantitative Definition Time/Space Complexity is O(n), O(log n), O(nlog n), O(n 2 ) Polynomial Growth of Computation Steps and/or memory requirement as the number of inputs increases Bad Algorithms Time/Space Complexity is O(k n ) where k is a constant. Exponential Growth of Computation Steps and/or Memory Requirement as the number of inputs increases If algorithms with exponential growth of complexity are only solutions to a problem, the problem is called a hard problem or intractable problem. Polynomial-time solvable problems are called easy problems or tractable problems. 12/13/02 Algorithm Theory 33 Growth of Functions n: number of inputs Unit: seconds if not specified Using more powerful computers is a good solution to hard problems? 12/13/02 Algorithm Theory 34

18 Computer Power Goes Up Steadily, and 750KHz The The Number of of Transistors Per Per Chip Chip Doubles Every Every Months! Pentium 4 2.4GHz B Tr Tr 4.3GHz 1+ GHz 12/13/02 Algorithm Theory 35 Source: Intel Yes, We Have Powerful Computers at Low Cost, but IBM System 360/50 DEC VAX11/780 Pentium 4 PC Performance ratio 0.15MIPS(1) Memory ratio 64KB(1) Price 360M yen Price/performance 1 1MIPS(6.7) 1MB(16) 72M yen 33 times 3792MIPS(25,280) 256MB(4,096) 250K yen More than 36M times! 12/13/02 Algorithm Theory 36

19 It s a mere drop in the bucket for hard problems Example For 4 algorithms A 1,A 2,A 3,A 4 whose complexities are O(n), O(n 2 ), O(n 100 ), O(2 n ), respectively, evaluate increases in input size when an 100 times faster computer is available. Assumptions A new computer M is 100 times faster than an old computer M. m x is a data size/sec on M with algorithm x m x is a data size/sec on M with algorithm x Algorithms Time increase in Input size Complexity M M' A 1 O(n) m 1 100m 1 A 2 O(n 2 ) m 2 10m 2 A 3 O(n 100 ) m m 3 A 4 O(2 n ) m 4 m /13/02 Algorithm Theory 37 Algorithm Analysis Example: MSP Problem Minimum Spanning Tree Problem For given graph G(V, E), find an acyclic subset T E that connects all of the vertices and whose total weight is minimized, where V is the set of vertices and E is the set of edges Weighed Graph Minimum Spanning Tree 12/13/02 Algorithm Theory 38

20 MST Algorithm (Kruskal s Algorithm) Step 1 Sort edges in an ascending order Step 2 Prepare v sets, each of which has each vertex as an entry, where v is the number of vertices. Step 3 Pick up an edge with the smallest weight. If two vertices of the edge belong to the different sets the edge is the edge of the spanning tree, and merge the two sets to which these two vertices belong. Otherwise, discard the edge, and check the next smallest edge. This step is repeated until all the edges have gone. 12/13/02 Algorithm Theory 39 Algorithm Behavior sets merge merge Initial Sets 1 st iteration 2 nd iteration merge Not changed Final MST weighted graph 3 rd iteration 4 th iteration 5 th iteration 12/13/02 Algorithm Theory 40

21 Complexity Analysis: MST Problem m E, n V where n < m n(n-1)/2 Computational complexity for sorting Sort m elements m log m Computational complexity for set operations One set operation for n elements log n (Find vertices connected by a given edge and merge two sets) m iterations of the set operation m log n Total Computation Complexity O(m log m) + O (m log n) = O(m log m) Space Complexity Memory for graph Working memory for set operations Total O(m) + O(n) O(m) O(m+n) O(m), O(n) 12/13/02 Algorithm Theory 41 Complexity Analysis: Optimization Problem For n+1 natural number (a 1, a 2, a 3,, a n, b), if there is a 0-1 vector (x 1, x 2,, x n ) that satisfies Σa i x i =b, output yes, otherwise no. Example: x=(3, 7, 5, 8, 2), b=11 Algorithm Represent x in n+1-bit number, and check throughout all the 0-1 combinations, e.g., from x=(000 0) to (111 1). If there is a combination that satisfies the condition, output yes, otherwise no. 12/13/02 Algorithm Theory 42

22 Complexity Analysis: Optimization Problem Number of iterations to check throughout to n+1 O( n ) Operation within one iteration: calculations for weighted sum and condition evaluation O(n) Total computation complexity O( n ) O(n) O(n n ) Hard Problem! Space complexity O(n) for storing each of vectors a and x Total Space Complexity O(n) 12/13/02 Algorithm Theory 43 Possible Solutions to Hard Problems Translate the hard problems to easy problems for which algorithms generating quasi-optimal solutions/output with polynomial complexity are available. Develop heuristic approaches that generate quasi-optimal solutions/output Neural Network Genetic algorithm Develop parallel algorithms on parallel computers (bruteforce!!) 12/13/02 Algorithm Theory 44

23 Neural Network Input Cell Output Inputs weight Output Biological Neuron Artificial Neuron Training an artificial neural network with pairs of inputs and their correct outputs, instead of programming Classification Pattern Recognition Associative Memory Problem class where there is no good deterministic algorithm!! 12/13/02 Algorithm Theory 45 Parallel Processing Big Problem Partitioning Subproblems Allocation Network Topologies P P P P: Processor Interconnection Network 12/13/02 Algorithm Theory 46

24 Trend in Supercomputer Performance 2x/year 12/13/02 Algorithm Theory 47 Single-CPU Performance, or Massive Parallelism? (As of Nov. 02) 1TFLOPS 10TFLOPS 100TFLOPS 10GFLOPS 1GFLOPS 100MFLOPS MasPar 100GFLOPS CM-2 NCube ASCI Red ASCI ASCI Purple in 05? white CP-PACS ASCI-Q ASCI Blue SP3 ES(SX-6) T3E Alpha Linux Cluster IntelDelta CM-5 SR2201 ORIGIN IBMpSeries SR8000 Cray/SGI SP P2SC (Tokyo) T3D SX-7(Tohoku) SX-5 (Osaka) InteliPSC SX-4 VPP5000 Cray/SGI Y-MP CRAY-3 12/13/02 Algorithm Theory 48 SX-2 SX-3 S-3800

25 Hard Problems are also Useful Example: Secure Networks with Cryptography Sender Receiver Plain text key Plain text key Encrypted text Encrypted Data Transmission Encrypted text Decryption without an encryption key leads to exponential computation complexity. Intrusion by someone else is very hard! E-Commerce Secure data exchange 12/13/02 Algorithm Theory 49 Summary Complexity analysis of algorithms gives a clue to judge whether the algorithms are efficient or not. Time Complexity Space Complexity Analysis of an algorithm is a cycle process of Analyzing Estimating Refining the analysis until an answer to the desired level of detail is reached Design efficient algorithms with polynomial-time/space complexity of O(n), O(log n), O(n log n), or O(n 2 )! 12/13/02 Algorithm Theory 50

Algorithms and Data Structures

Algorithm Analysis Page 1 BFH-TI: Softwareschule Schweiz Algorithm Analysis Dr. CAS SD01 Algorithm Analysis Page 2 Outline Course and Textbook Overview Analysis of Algorithm Pseudo-Code and Primitive Operations

Efficiency of algorithms. Algorithms. Efficiency of algorithms. Binary search and linear search. Best, worst and average case.

Algorithms Efficiency of algorithms Computational resources: time and space Best, worst and average case performance How to compare algorithms: machine-independent measure of efficiency Growth rate Complexity

Why? A central concept in Computer Science. Algorithms are ubiquitous.

Analysis of Algorithms: A Brief Introduction Why? A central concept in Computer Science. Algorithms are ubiquitous. Using the Internet (sending email, transferring files, use of search engines, online

Data Structures. Algorithm Performance and Big O Analysis

Data Structures Algorithm Performance and Big O Analysis What s an Algorithm? a clearly specified set of instructions to be followed to solve a problem. In essence: A computer program. In detail: Defined

Running Time ( 3.1) Analysis of Algorithms. Experimental Studies ( 3.1.1) Limitations of Experiments. Pseudocode ( 3.1.2) Theoretical Analysis

Ruig Time ( 3.) Aalysis of Algorithms Iput Algorithm Output A algorithm is a step-by-step procedure for solvig a problem i a fiite amout of time. Most algorithms trasform iput objects ito output objects.

SIMS 255 Foundations of Software Design. Complexity and NP-completeness

SIMS 255 Foundations of Software Design Complexity and NP-completeness Matt Welsh November 29, 2001 mdw@cs.berkeley.edu 1 Outline Complexity of algorithms Space and time complexity ``Big O'' notation Complexity

In mathematics, it is often important to get a handle on the error term of an approximation. For instance, people will write

Big O notation (with a capital letter O, not a zero), also called Landau's symbol, is a symbolism used in complexity theory, computer science, and mathematics to describe the asymptotic behavior of functions.

CSE373: Data Structures and Algorithms Lecture 3: Math Review; Algorithm Analysis. Linda Shapiro Winter 2015

CSE373: Data Structures and Algorithms Lecture 3: Math Review; Algorithm Analysis Linda Shapiro Today Registration should be done. Homework 1 due 11:59 pm next Wednesday, January 14 Review math essential

CSC148 Lecture 8. Algorithm Analysis Binary Search Sorting

CSC148 Lecture 8 Algorithm Analysis Binary Search Sorting Algorithm Analysis Recall definition of Big Oh: We say a function f(n) is O(g(n)) if there exists positive constants c,b such that f(n)

MapReduce and Distributed Data Analysis. Sergei Vassilvitskii Google Research

MapReduce and Distributed Data Analysis Google Research 1 Dealing With Massive Data 2 2 Dealing With Massive Data Polynomial Memory Sublinear RAM Sketches External Memory Property Testing 3 3 Dealing With

Sorting revisited. Build the binary search tree: O(n^2) Traverse the binary tree: O(n) Total: O(n^2) + O(n) = O(n^2)

Sorting revisited How did we use a binary search tree to sort an array of elements? Tree Sort Algorithm Given: An array of elements to sort 1. Build a binary search tree out of the elements 2. Traverse

Today s Outline. Exercise. Binary Search Analysis. Linear Search Analysis. Asymptotic Analysis. Analyzing Code. Announcements. Asymptotic Analysis

Today s Outline Announcements Assignment #1 due Thurs, Oct 7 at 11:45pm Asymptotic Analysis Asymptotic Analysis CSE 7 Data Structures & Algorithms Ruth Anderson Autumn 2010 Exercise Analyzing Code bool

Computer Algorithms. NP-Complete Problems. CISC 4080 Yanjun Li

Computer Algorithms NP-Complete Problems NP-completeness The quest for efficient algorithms is about finding clever ways to bypass the process of exhaustive search, using clues from the input in order

Class Overview. CSE 326: Data Structures. Goals. Goals. Data Structures. Goals. Introduction

Class Overview CSE 326: Data Structures Introduction Introduction to many of the basic data structures used in computer software Understand the data structures Analyze the algorithms that use them Know

Zabin Visram Room CS115 CS126 Searching. Binary Search

Zabin Visram Room CS115 CS126 Searching Binary Search Binary Search Sequential search is not efficient for large lists as it searches half the list, on average Another search algorithm Binary search Very

Complexity Theory. IE 661: Scheduling Theory Fall 2003 Satyaki Ghosh Dastidar

Complexity Theory IE 661: Scheduling Theory Fall 2003 Satyaki Ghosh Dastidar Outline Goals Computation of Problems Concepts and Definitions Complexity Classes and Problems Polynomial Time Reductions Examples

CSE 326, Data Structures. Sample Final Exam. Problem Max Points Score 1 14 (2x7) 2 18 (3x6) 3 4 4 7 5 9 6 16 7 8 8 4 9 8 10 4 Total 92.

Name: Email ID: CSE 326, Data Structures Section: Sample Final Exam Instructions: The exam is closed book, closed notes. Unless otherwise stated, N denotes the number of elements in the data structure

CSC 180 H1F Algorithm Runtime Analysis Lecture Notes Fall 2015

1 Introduction These notes introduce basic runtime analysis of algorithms. We would like to be able to tell if a given algorithm is time-efficient, and to be able to compare different algorithms. 2 Linear

AP Computer Science AB Syllabus 1

AP Computer Science AB Syllabus 1 Course Resources Java Software Solutions for AP Computer Science, J. Lewis, W. Loftus, and C. Cocking, First Edition, 2004, Prentice Hall. Video: Sorting Out Sorting,

Theoretical Computer Science (Bridging Course) Complexity

Theoretical Computer Science (Bridging Course) Complexity Gian Diego Tipaldi A scenario You are a programmer working for a logistics company Your boss asks you to implement a program that optimizes the

Binary Search. Search for x in a sorted array A.

Divide and Conquer A general paradigm for algorithm design; inspired by emperors and colonizers. Three-step process: 1. Divide the problem into smaller problems. 2. Conquer by solving these problems. 3.

Algorithms, Integers

CHAPTER 3 Algorithms, Integers 3.1. Algorithms Consider the following list of instructions to find the maximum of three numbers a, b, c: 1. Assign variable x the value of a. 2. If b > x then assign x the

Approximation Algorithms

Approximation Algorithms or: How I Learned to Stop Worrying and Deal with NP-Completeness Ong Jit Sheng, Jonathan (A0073924B) March, 2012 Overview Key Results (I) General techniques: Greedy algorithms

2.3 Scheduling jobs on identical parallel machines

2.3 Scheduling jobs on identical parallel machines There are jobs to be processed, and there are identical machines (running in parallel) to which each job may be assigned Each job = 1,,, must be processed

CPSC 211 Data Structures & Implementations (c) Texas A&M University [ 313]

CPSC 211 Data Structures & Implementations (c) Texas A&M University [ 313] File Structures A file is a collection of data stored on mass storage (e.g., disk or tape) Why on mass storage? too big to fit

CS/COE 1501 http://cs.pitt.edu/~bill/1501/

CS/COE 1501 http://cs.pitt.edu/~bill/1501/ Lecture 01 Course Introduction Meta-notes These notes are intended for use by students in CS1501 at the University of Pittsburgh. They are provided free of charge

A Performance Study of Load Balancing Strategies for Approximate String Matching on an MPI Heterogeneous System Environment

A Performance Study of Load Balancing Strategies for Approximate String Matching on an MPI Heterogeneous System Environment Panagiotis D. Michailidis and Konstantinos G. Margaritis Parallel and Distributed

Cost Model: Work, Span and Parallelism. 1 The RAM model for sequential computation:

CSE341T 08/31/2015 Lecture 3 Cost Model: Work, Span and Parallelism In this lecture, we will look at how one analyze a parallel program written using Cilk Plus. When we analyze the cost of an algorithm

Analysis of Binary Search algorithm and Selection Sort algorithm

Analysis of Binary Search algorithm and Selection Sort algorithm In this section we shall take up two representative problems in computer science, work out the algorithms based on the best strategy to

Near Optimal Solutions

Near Optimal Solutions Many important optimization problems are lacking efficient solutions. NP-Complete problems unlikely to have polynomial time solutions. Good heuristics important for such problems.

The Union-Find Problem Kruskal s algorithm for finding an MST presented us with a problem in data-structure design. As we looked at each edge,

The Union-Find Problem Kruskal s algorithm for finding an MST presented us with a problem in data-structure design. As we looked at each edge, cheapest first, we had to determine whether its two endpoints

The Running Time of Programs

CHAPTER 3 The Running Time of Programs In Chapter 2, we saw two radically different algorithms for sorting: selection sort and merge sort. There are, in fact, scores of algorithms for sorting. This situation

Introduction to Algorithms Review information for Prelim 1 CS 4820, Spring 2010 Distributed Wednesday, February 24

Introduction to Algorithms Review information for Prelim 1 CS 4820, Spring 2010 Distributed Wednesday, February 24 The final exam will cover seven topics. 1. greedy algorithms 2. divide-and-conquer algorithms

Analysis of Algorithms: The Non-recursive Case

Analysis of Algorithms: The Non-recursive Case Key topics: * Introduction * Generalizing Running Time * Doing a Timing Analysis * Big-Oh Notation * Big-Oh Operations * Analyzing Some Simple Programs -

Discuss the size of the instance for the minimum spanning tree problem.

3.1 Algorithm complexity The algorithms A, B are given. The former has complexity O(n 2 ), the latter O(2 n ), where n is the size of the instance. Let n A 0 be the size of the largest instance that can

Algorithms. Margaret M. Fleck. 18 October 2010

Algorithms Margaret M. Fleck 18 October 2010 These notes cover how to analyze the running time of algorithms (sections 3.1, 3.3, 4.4, and 7.1 of Rosen). 1 Introduction The main reason for studying big-o

6.080 / 6.089 Great Ideas in Theoretical Computer Science Spring 2008

MIT OpenCourseWare http://ocw.mit.edu 6.080 / 6.089 Great Ideas in Theoretical Computer Science Spring 008 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

EFFICIENT EXTERNAL SORTING ON FLASH MEMORY EMBEDDED DEVICES

ABSTRACT EFFICIENT EXTERNAL SORTING ON FLASH MEMORY EMBEDDED DEVICES Tyler Cossentine and Ramon Lawrence Department of Computer Science, University of British Columbia Okanagan Kelowna, BC, Canada tcossentine@gmail.com

Classification - Examples

Lecture 2 Scheduling 1 Classification - Examples 1 r j C max given: n jobs with processing times p 1,...,p n and release dates r 1,...,r n jobs have to be scheduled without preemption on one machine taking

Integer Factorization using the Quadratic Sieve

Integer Factorization using the Quadratic Sieve Chad Seibert* Division of Science and Mathematics University of Minnesota, Morris Morris, MN 56567 seib0060@morris.umn.edu March 16, 2011 Abstract We give

The Classes P and NP. mohamed@elwakil.net

Intractable Problems The Classes P and NP Mohamed M. El Wakil mohamed@elwakil.net 1 Agenda 1. What is a problem? 2. Decidable or not? 3. The P class 4. The NP Class 5. TheNP Complete class 2 What is a

IMPROVING PERFORMANCE OF RANDOMIZED SIGNATURE SORT USING HASHING AND BITWISE OPERATORS

Volume 2, No. 3, March 2011 Journal of Global Research in Computer Science RESEARCH PAPER Available Online at www.jgrcs.info IMPROVING PERFORMANCE OF RANDOMIZED SIGNATURE SORT USING HASHING AND BITWISE

Union-Find Algorithms. network connectivity quick find quick union improvements applications

Union-Find Algorithms network connectivity quick find quick union improvements applications 1 Subtext of today s lecture (and this course) Steps to developing a usable algorithm. Define the problem. Find

CMPS 102 Solutions to Homework 1

CMPS 0 Solutions to Homework Lindsay Brown, lbrown@soe.ucsc.edu September 9, 005 Problem..- p. 3 For inputs of size n insertion sort runs in 8n steps, while merge sort runs in 64n lg n steps. For which

CSC2420 Fall 2012: Algorithm Design, Analysis and Theory

CSC2420 Fall 2012: Algorithm Design, Analysis and Theory Allan Borodin November 15, 2012; Lecture 10 1 / 27 Randomized online bipartite matching and the adwords problem. We briefly return to online algorithms

CAD Algorithms The Classes P and NP Mohammad Tehranipoor ECE Department 6 September 2010 1 P and NP P and NP are two families of problems. P is a class which contains all of the problems we solve using

Distance Degree Sequences for Network Analysis

Universität Konstanz Computer & Information Science Algorithmics Group 15 Mar 2005 based on Palmer, Gibbons, and Faloutsos: ANF A Fast and Scalable Tool for Data Mining in Massive Graphs, SIGKDD 02. Motivation

Molecular Computing. david.wishart@ualberta.ca 3-41 Athabasca Hall Sept. 30, 2013

Molecular Computing david.wishart@ualberta.ca 3-41 Athabasca Hall Sept. 30, 2013 What Was The World s First Computer? The World s First Computer? ENIAC - 1946 Antikythera Mechanism - 80 BP Babbage Analytical

External Sorting. Why Sort? 2-Way Sort: Requires 3 Buffers. Chapter 13

External Sorting Chapter 13 Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1 Why Sort? A classic problem in computer science! Data requested in sorted order e.g., find students in increasing

Applied Algorithm Design Lecture 5

Applied Algorithm Design Lecture 5 Pietro Michiardi Eurecom Pietro Michiardi (Eurecom) Applied Algorithm Design Lecture 5 1 / 86 Approximation Algorithms Pietro Michiardi (Eurecom) Applied Algorithm Design

NP-completeness and the real world. NP completeness. NP-completeness and the real world (2) NP-completeness and the real world

-completeness and the real world completeness Course Discrete Biological Models (Modelli Biologici Discreti) Zsuzsanna Lipták Imagine you are working for a biotech company. One day your boss calls you

Diagonalization. Ahto Buldas. Lecture 3 of Complexity Theory October 8, 2009. Slides based on S.Aurora, B.Barak. Complexity Theory: A Modern Approach.

Diagonalization Slides based on S.Aurora, B.Barak. Complexity Theory: A Modern Approach. Ahto Buldas Ahto.Buldas@ut.ee Background One basic goal in complexity theory is to separate interesting complexity

Randomized algorithms

Randomized algorithms March 10, 2005 1 What are randomized algorithms? Algorithms which use random numbers to make decisions during the executions of the algorithm. Why would we want to do this?? Deterministic

Exponential time algorithms for graph coloring

Exponential time algorithms for graph coloring Uriel Feige Lecture notes, March 14, 2011 1 Introduction Let [n] denote the set {1,..., k}. A k-labeling of vertices of a graph G(V, E) is a function V [k].

Distributed Computing over Communication Networks: Maximal Independent Set

Distributed Computing over Communication Networks: Maximal Independent Set What is a MIS? MIS An independent set (IS) of an undirected graph is a subset U of nodes such that no two nodes in U are adjacent.

Faster deterministic integer factorisation

David Harvey (joint work with Edgar Costa, NYU) University of New South Wales 25th October 2011 The obvious mathematical breakthrough would be the development of an easy way to factor large prime numbers

Classification - Examples -1- 1 r j C max given: n jobs with processing times p 1,..., p n and release dates

Lecture 2 Scheduling 1 Classification - Examples -1-1 r j C max given: n jobs with processing times p 1,..., p n and release dates r 1,..., r n jobs have to be scheduled without preemption on one machine

4.2 Sorting and Searching

Sequential Search: Java Implementation 4.2 Sorting and Searching Scan through array, looking for key. search hit: return array index search miss: return -1 public static int search(string key, String[]

Boolean Network Models

Boolean Network Models 2/5/03 History Kaufmann, 1970s Studied organization and dynamics properties of (N,k) Boolean Networks Found out that highly connected networks behave differently than lowly connected

Lecture 4: Properties of and Rules for

Lecture 4: Properties of and Rules for Asymptotic Big-Oh, Big-Omega, and Big-Theta Notation Georgy Gimel farb COMPSCI 220 Algorithms and Data Structures 1 / 13 1 Time complexity 2 Big-Oh rules Scaling

The Conference Call Search Problem in Wireless Networks

The Conference Call Search Problem in Wireless Networks Leah Epstein 1, and Asaf Levin 2 1 Department of Mathematics, University of Haifa, 31905 Haifa, Israel. lea@math.haifa.ac.il 2 Department of Statistics,

Analysis of Algorithms, I

Analysis of Algorithms, I CSOR W4231.002 Eleni Drinea Computer Science Department Columbia University Thursday, February 26, 2015 Outline 1 Recap 2 Representing graphs 3 Breadth-first search (BFS) 4 Applications

Data Structures and Algorithms Written Examination

Data Structures and Algorithms Written Examination 22 February 2013 FIRST NAME STUDENT NUMBER LAST NAME SIGNATURE Instructions for students: Write First Name, Last Name, Student Number and Signature where

Page 1. CSCE 310J Data Structures & Algorithms. CSCE 310J Data Structures & Algorithms. P, NP, and NP-Complete. Polynomial-Time Algorithms

CSCE 310J Data Structures & Algorithms P, NP, and NP-Complete Dr. Steve Goddard goddard@cse.unl.edu CSCE 310J Data Structures & Algorithms Giving credit where credit is due:» Most of the lecture notes

Using Data-Oblivious Algorithms for Private Cloud Storage Access. Michael T. Goodrich Dept. of Computer Science

Using Data-Oblivious Algorithms for Private Cloud Storage Access Michael T. Goodrich Dept. of Computer Science Privacy in the Cloud Alice owns a large data set, which she outsources to an honest-but-curious

Hardware: Input, Processing, and Output Devices

Hardware: Input, Processing, and Output Devices Computer Systems Hardware Components Execution of an Instruction Processing Characteristics and Functions Physical Characteristics of CPU Memory Characteristics

1. Nondeterministically guess a solution (called a certificate) 2. Check whether the solution solves the problem (called verification)

Some N P problems Computer scientists have studied many N P problems, that is, problems that can be solved nondeterministically in polynomial time. Traditionally complexity question are studied as languages:

Big Data Systems CS 5965/6965 FALL 2015

Big Data Systems CS 5965/6965 FALL 2015 Today General course overview Expectations from this course Q&A Introduction to Big Data Assignment #1 General Course Information Course Web Page http://www.cs.utah.edu/~hari/teaching/fall2015.html

Computer Graphics Hardware An Overview

Computer Graphics Hardware An Overview Graphics System Monitor Input devices CPU/Memory GPU Raster Graphics System Raster: An array of picture elements Based on raster-scan TV technology The screen (and

Many algorithms, particularly divide and conquer algorithms, have time complexities which are naturally

Recurrence Relations Many algorithms, particularly divide and conquer algorithms, have time complexities which are naturally modeled by recurrence relations. A recurrence relation is an equation which

! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. #-approximation algorithm.

Approximation Algorithms 11 Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of three

A Comparison of General Approaches to Multiprocessor Scheduling

A Comparison of General Approaches to Multiprocessor Scheduling Jing-Chiou Liou AT&T Laboratories Middletown, NJ 0778, USA jing@jolt.mt.att.com Michael A. Palis Department of Computer Science Rutgers University

Topics. Introduction. Java History CS 146. Introduction to Programming and Algorithms Module 1. Module Objectives

Introduction to Programming and Algorithms Module 1 CS 146 Sam Houston State University Dr. Tim McGuire Module Objectives To understand: the necessity of programming, differences between hardware and software,

Merge Sort. 2004 Goodrich, Tamassia. Merge Sort 1

Merge Sort 7 2 9 4 2 4 7 9 7 2 2 7 9 4 4 9 7 7 2 2 9 9 4 4 Merge Sort 1 Divide-and-Conquer Divide-and conquer is a general algorithm design paradigm: Divide: divide the input data S in two disjoint subsets

SIGMOD RWE Review Towards Proximity Pattern Mining in Large Graphs

SIGMOD RWE Review Towards Proximity Pattern Mining in Large Graphs Fabian Hueske, TU Berlin June 26, 21 1 Review This document is a review report on the paper Towards Proximity Pattern Mining in Large

1 Approximating Set Cover

CS 05: Algorithms (Grad) Feb 2-24, 2005 Approximating Set Cover. Definition An Instance (X, F ) of the set-covering problem consists of a finite set X and a family F of subset of X, such that every elemennt

Chapter 6: Episode discovery process

Chapter 6: Episode discovery process Algorithmic Methods of Data Mining, Fall 2005, Chapter 6: Episode discovery process 1 6. Episode discovery process The knowledge discovery process KDD process of analyzing

Introduction to Algorithms. Part 3: P, NP Hard Problems

Introduction to Algorithms Part 3: P, NP Hard Problems 1) Polynomial Time: P and NP 2) NP-Completeness 3) Dealing with Hard Problems 4) Lower Bounds 5) Books c Wayne Goddard, Clemson University, 2004 Chapter

Performance Metrics and Scalability Analysis. Performance Metrics and Scalability Analysis

Performance Metrics and Scalability Analysis 1 Performance Metrics and Scalability Analysis Lecture Outline Following Topics will be discussed Requirements in performance and cost Performance metrics Work

Complexity Classes P and NP

Complexity Classes P and NP MATH 3220 Supplemental Presentation by John Aleshunas The cure for boredom is curiosity. There is no cure for curiosity Dorothy Parker Computational Complexity Theory In computer

Small Maximal Independent Sets and Faster Exact Graph Coloring

Small Maximal Independent Sets and Faster Exact Graph Coloring David Eppstein Univ. of California, Irvine Dept. of Information and Computer Science The Exact Graph Coloring Problem: Given an undirected

Algorithms, Flowcharts & Program Design. ComPro

Algorithms, Flowcharts & Program Design ComPro Definition Algorithm: o sequence of steps to be performed in order to solve a problem by the computer. Flowchart: o graphical or symbolic representation of

Rethinking SIMD Vectorization for In-Memory Databases

SIGMOD 215, Melbourne, Victoria, Australia Rethinking SIMD Vectorization for In-Memory Databases Orestis Polychroniou Columbia University Arun Raghavan Oracle Labs Kenneth A. Ross Columbia University Latest

A Note on Maximum Independent Sets in Rectangle Intersection Graphs

A Note on Maximum Independent Sets in Rectangle Intersection Graphs Timothy M. Chan School of Computer Science University of Waterloo Waterloo, Ontario N2L 3G1, Canada tmchan@uwaterloo.ca September 12,

Chapter 20: Data Analysis

Chapter 20: Data Analysis Database System Concepts, 6 th Ed. See www.db-book.com for conditions on re-use Chapter 20: Data Analysis Decision Support Systems Data Warehousing Data Mining Classification

Eastern Washington University Department of Computer Science. Questionnaire for Prospective Masters in Computer Science Students

Eastern Washington University Department of Computer Science Questionnaire for Prospective Masters in Computer Science Students I. Personal Information Name: Last First M.I. Mailing Address: Permanent

Binary Heaps * * * * * * * / / \ / \ / \ / \ / \ * * * * * * * * * * * / / \ / \ / / \ / \ * * * * * * * * * *

Binary Heaps A binary heap is another data structure. It implements a priority queue. Priority Queue has the following operations: isempty add (with priority) remove (highest priority) peek (at highest

5.1 Bipartite Matching

CS787: Advanced Algorithms Lecture 5: Applications of Network Flow In the last lecture, we looked at the problem of finding the maximum flow in a graph, and how it can be efficiently solved using the Ford-Fulkerson

The Classes P and NP

The Classes P and NP We now shift gears slightly and restrict our attention to the examination of two families of problems which are very important to computer scientists. These families constitute the

8.1 Min Degree Spanning Tree

CS880: Approximations Algorithms Scribe: Siddharth Barman Lecturer: Shuchi Chawla Topic: Min Degree Spanning Tree Date: 02/15/07 In this lecture we give a local search based algorithm for the Min Degree

Answers to some of the exercises.

Answers to some of the exercises. Chapter 2. Ex.2.1 (a) There are several ways to do this. Here is one possibility. The idea is to apply the k-center algorithm first to D and then for each center in D

THE SCHEDULING OF MAINTENANCE SERVICE

THE SCHEDULING OF MAINTENANCE SERVICE Shoshana Anily Celia A. Glass Refael Hassin Abstract We study a discrete problem of scheduling activities of several types under the constraint that at most a single

The Taxman Game. Robert K. Moniot September 5, 2003

The Taxman Game Robert K. Moniot September 5, 2003 1 Introduction Want to know how to beat the taxman? Legally, that is? Read on, and we will explore this cute little mathematical game. The taxman game

Scheduling Shop Scheduling. Tim Nieberg

Scheduling Shop Scheduling Tim Nieberg Shop models: General Introduction Remark: Consider non preemptive problems with regular objectives Notation Shop Problems: m machines, n jobs 1,..., n operations

ALGEBRA. sequence, term, nth term, consecutive, rule, relationship, generate, predict, continue increase, decrease finite, infinite

ALGEBRA Pupils should be taught to: Generate and describe sequences As outcomes, Year 7 pupils should, for example: Use, read and write, spelling correctly: sequence, term, nth term, consecutive, rule,

JUST-IN-TIME SCHEDULING WITH PERIODIC TIME SLOTS. Received December May 12, 2003; revised February 5, 2004

Scientiae Mathematicae Japonicae Online, Vol. 10, (2004), 431 437 431 JUST-IN-TIME SCHEDULING WITH PERIODIC TIME SLOTS Ondřej Čepeka and Shao Chin Sung b Received December May 12, 2003; revised February