Fundamentals of Computer Science

Transcription

1 Fundamentals of Computer Science Chapter 13: Complexity theory Henrik Björklund Umeå University March 17, 2014

2 Moore s law

3 Moore s law The famous claim known as Moore s law states that the capacity of computer hardware doubles every 1.5 years.

4 Moore s law The famous claim known as Moore s law states that the capacity of computer hardware doubles every 1.5 years. If Moore s law holds true, in 20 years time, computers will be about 3300 times faster than today.

5 Moore s law The famous claim known as Moore s law states that the capacity of computer hardware doubles every 1.5 years. If Moore s law holds true, in 20 years time, computers will be about 3300 times faster than today. Suppose that we have an algorithm that runs in time 2 n, where n is the size of its input. How much larger instances of this problem will we be able to run in 20 years?

6 Complexity theory Complexity theory is the study of how hard computational problems are.

7 Complexity theory Complexity theory is the study of how hard computational problems are. To be able to achieve this, we have to reason abstractly, mostly disregarding such things as the setup of individual computers, programming languages, etc.

8 Complexity theory Complexity theory is the study of how hard computational problems are. To be able to achieve this, we have to reason abstractly, mostly disregarding such things as the setup of individual computers, programming languages, etc. What we are really interested are the inherent features of computational problems.

9 Complexity theory Complexity theory is the study of how hard computational problems are. To be able to achieve this, we have to reason abstractly, mostly disregarding such things as the setup of individual computers, programming languages, etc. What we are really interested are the inherent features of computational problems. We are particularly interested in the growth rate of functions and are mostly satisfied with getting the order of magnitude right.

10 Asymptotic notation Let f : N N and g : N N be two functions. If there is a positive constant c and a number n 0 such that f (n) c g(n) for all n > n 0, we say that f has order at most g and write f (n) = O(g(n)).

11 Asymptotic notation Let f : N N and g : N N be two functions. If there is a positive constant c and a number n 0 such that f (n) c g(n) for all n > n 0, we say that f has order at least g and write f (n) = Ω(g(n)).

12 Asymptotic notation Let f : N N and g : N N be two functions. If there are positive constants c 1 and c 2 and a number n 0 such that c 1 g(n) f (n) c 2 g(n) for all n > n 0, we say that f has order g and write f (n) = Θ(g(n)).

13 Efficiency of computations Efficiency is measured by resource requriements, such as time and space. We therefore talk about time-complexity and space complexity.

14 Efficiency of computations Efficiency is measured by resource requriements, such as time and space. We therefore talk about time-complexity and space complexity. Time-complexity is a rough measure of how long time it takes to solve a particular problem on a computer.

15 Efficiency of computations Efficiency is measured by resource requriements, such as time and space. We therefore talk about time-complexity and space complexity. Time-complexity is a rough measure of how long time it takes to solve a particular problem on a computer. How long does it take to sort a list of integers?

16 Efficiency of computations Efficiency is measured by resource requriements, such as time and space. We therefore talk about time-complexity and space complexity. Time-complexity is a rough measure of how long time it takes to solve a particular problem on a computer. How long does it take to sort a list of integers? Depents on the number of items, the computer used, the algorithm used, how the algorithm is written, etc.

17 Efficiency of computations n order to get anything resembling a general picture, we need to focus on the fundamentals: As a computer model we use the Turing machine. The size of the problem instance will be denoted by n. In analyzing an algorithm, we study how the resource requrements of the algorithm grows with n.

18 Efficiency of computations n order to get anything resembling a general picture, we need to focus on the fundamentals: As a computer model we use the Turing machine. The size of the problem instance will be denoted by n. In analyzing an algorithm, we study how the resource requrements of the algorithm grows with n. Goal (time complexity): Characterize the time required to solve instances of a problem as a function of their sizes, using a Turing machine as the computer model.

19 Turing machines and time We think of Turing machines as making one move per time unit, so we count the number of moves.

20 Turing machines and time We think of Turing machines as making one move per time unit, so we count the number of moves. We are primarily interested in worst case resource requirements. We say that an algorithm has time complexity at most T (n) if, for any input of size n, the algorithm halts in at most T (n) steps.

21 Turing machines and complexity Unlike the decidability of a problem, its time complexity can be affected by the choice of Turing machine model, in particular whether the model is deterministic or non-deterministic and how many tapes the machines have.

22 Turing machines and complexity Unlike the decidability of a problem, its time complexity can be affected by the choice of Turing machine model, in particular whether the model is deterministic or non-deterministic and how many tapes the machines have. Example: Consder the language {a n b n n N}. A single-tape TM for this problem needs Ω(n 2 ) moves. A two-tape TM, on the other hand, needs only O(n) moves.

23 Nondeterminism Example: Consider the CFG membership problem, i.e., we are given a context-free grammar G and a word w and are asked to decide whether w L(G).

24 Nondeterminism Example: Consider the CFG membership problem, i.e., we are given a context-free grammar G and a word w and are asked to decide whether w L(G). A deterministic algorithm, such as CYK, needs Ω(n 3 ) moves.

25 Nondeterminism Example: Consider the CFG membership problem, i.e., we are given a context-free grammar G and a word w and are asked to decide whether w L(G). A deterministic algorithm, such as CYK, needs Ω(n 3 ) moves. A nondeterministic algorithm can guess what rules are used in the shortest derivation of w in G. Since the length of the shortest derivation is linear in w, such an algorithm needs O(n) steps.

26 Propositional formulas We consider the following definition of propsitional logic: A Boolean variable is one that can take exactly two values, true and false or, equivalently 1 or 0. A Boolean constant is either 1 or 0. Boolean operators are used to construct Boolean expressions or propositional formulas from variables and constants. We consider only the Boolean operators (or) (and) (negation)

27 Conjunctive normal form A literal is a variable x or a negation of a variable x.

28 Conjunctive normal form A literal is a variable x or a negation of a variable x. A clause is a disjunction of a number of literals, e.g., (x 1 x 2 x 3 x 4 ).

29 Conjunctive normal form A literal is a variable x or a negation of a variable x. A clause is a disjunction of a number of literals, e.g., (x 1 x 2 x 3 x 4 ). A formula on conjunctive normal form (CNF) is a conjunction of a number of clauses. If C 1, C 2,..., C m are clauses, then is a CNF formula. C 1 C 2 C m

30 The SAT problem The SAT problem is the satisfiability problem for CNF formulas, i.e., we are given a formula φ on CNF and are asked if it is satisfiable, i.e., whether there exists an assignment of Boolean values to all variables in φ that makes φ true.

31 SAT and TM models The following deterministic algorithm for SAT is trivial: Let φ be the input formula and x 1,..., x n the variables that appear in φ. For every possible assignment of Boolean values to x 1,..., x n, check if it makes φ true. The algorithm has exponential time complexity, since there are 2 n possible variable assignments.

32 SAT and TM models The following deterministic algorithm for SAT is trivial: Let φ be the input formula and x 1,..., x n the variables that appear in φ. For every possible assignment of Boolean values to x 1,..., x n, check if it makes φ true. The algorithm has exponential time complexity, since there are 2 n possible variable assignments. There is an even simpler nondeterministic algorithm: If φ is satisfiable, guess an assignment and verify that it satisfies φ. This algorithm is essentially O(n).

33 Complexity classes Definition. A language L is a member of the class DTIME(T (n)) if there is a deterministic multitape Turing machine that decides L in time O(T (n)).

34 Complexity classes Definition. A language L is a member of the class DTIME(T (n)) if there is a deterministic multitape Turing machine that decides L in time O(T (n)). Definition. A language L is a member of the class NTIME(T (n)) if there is a nondeterministic multitape Turing machine that decides L in time O(T (n)).

35 Complexity classes Definition. A language L is a member of the class DTIME(T (n)) if there is a deterministic multitape Turing machine that decides L in time O(T (n)). Definition. A language L is a member of the class NTIME(T (n)) if there is a nondeterministic multitape Turing machine that decides L in time O(T (n)). Example relation: For every integer k 1, DTIME(n k ) DTIME(n k+1 ).

36 The class P The difference between DTIME(n k ) and DTIME(n k+1 ) at least partly depends on the details of the computational model.

37 The class P The difference between DTIME(n k ) and DTIME(n k+1 ) at least partly depends on the details of the computational model. This leads us to disregard this difference and introduce the rock star of all complexity classes: P = i 1 DTIME(n i ) This class includes all languages that have a deterministic Turing machine that decides them in polynomial time, no matter what the degree of the polynomial is.

38 The class NP Symmetrically, we can define the class of all languages that have a nondeterministic Turing machine that decides them in polynomial time, no matter what the degree of the polynomial is: NP = i 1 NTIME(n i )

39 P and tractability The interest in the class P stems largely from the following facts: It is robust in the sense that whatever reasonable deterministic model of computation we use to define P, we end up with the same class. P is often viewed as more or less synonymous with tractability. This is because most natural problems that have been proved to belong to P actually belong to the lower levels of P, i.e., DTIME(n), DTIME(n 2 ), DTIME(n 3 ).

40 The million dollar question It is obvious that P NP, but it is not known whether the inclusion is proper or, in other words, whether P = NP.

41 The million dollar question It is obvious that P NP, but it is not known whether the inclusion is proper or, in other words, whether P = NP. This is one of seven Millenium prize problems selected by the Clay Mathematics Institute.

42 P vs NP To prove that P = NP, one has to be able to argue that every problem in NP can be solved by some deterministic Turing machine in polynomial time.

43 P vs NP To prove that P = NP, one has to be able to argue that every problem in NP can be solved by some deterministic Turing machine in polynomial time. To prove that P NP, one has to be able to argue that there is some problem in NP for which there is no deterministic Turing machine that solves it in polynomial time.

44 A closer look at non-determinism A nondeterministic TM M decides a language L in time T (n) if for every word w, there is some computational path that halts and accepts if and only if w L and, furthermore, the shortest such path is at most T ( w ) steps long.

45 A closer look at non-determinism A nondeterministic TM M decides a language L in time T (n) if for every word w, there is some computational path that halts and accepts if and only if w L and, furthermore, the shortest such path is at most T ( w ) steps long. A polynomial time nondeterministic algorithm can be seen as operating in two steps: 1. Nondeterministically guess a solution (certificate) 2. Check that the solution really is a solution (verification)

46 CLIQUE

47 CLIQUE

48 Polynomial time reductions Definition. A language L 1 over Σ is polynomial time reducible to a language L 2 over Γ if there exists a deterministic Turing machine M that computes a function f : Σ Γ such that M runs in polynomial time and w L 1 f (w) L 2.

49 Conclusions from reductions Just as with general reductions, we can draw two types of conclusions from a polynomial time reduction, depending on what we already know about the languages involved.

50 Conclusions from reductions Just as with general reductions, we can draw two types of conclusions from a polynomial time reduction, depending on what we already know about the languages involved. Assume that L 1 is plolynomial time reducible to L 2.

51 Conclusions from reductions Just as with general reductions, we can draw two types of conclusions from a polynomial time reduction, depending on what we already know about the languages involved. Assume that L 1 is plolynomial time reducible to L If L 2 belongs to P, then so does L 1.

52 Conclusions from reductions Just as with general reductions, we can draw two types of conclusions from a polynomial time reduction, depending on what we already know about the languages involved. Assume that L 1 is plolynomial time reducible to L If L 2 belongs to P, then so does L If L 1 does not belong to P, then neither does L 2.