Handout: How to calculate time complexity? CSE 101 Winter 2014

Transcription

1 Hadout: How to calculate time complexity? CSE 101 Witer 014 Recipe (a) Kow algorithm If you are usig a modied versio of a kow algorithm, you ca piggyback your aalysis o the complexity of the origial algorithm For example, if you use a modied versio of DFS or Dijkstra's algorithm, as log as your modicatios do ot aect the origial ruig time, you ca use the complexity of these other algorithms However, you must state (ad be correct) about why your chages do ot aect the origial algorithm's ruig time For example: The algorithm performs DFS ad marks each vertex with a color whe it is visited This is a costat amout of additioal work per vertex ad so the ruig time remais the same as DFS: O(jV j + jej) (b) Master Theorem If the time complexity equatio ca be writte as follow, the we ca use the Master Theorem T () = T () = a T b + O( d ) If log b a < d; T () = O( d ) If log b a = d; T () = O( d log ) If log b a > d; T () = O( log b a ) (c) Else, we have to evaluate T () to be able to take the big-o out of it (see Evaluatig sums ad Examples) There are 3 ways of doig this: Guess by lookig at the rst terms the prove, Guess by urollig the equatio the prove, Ad look at the recursio tree Importat trick Whe aalysig recurreces, for coveiece you ca 1 assume is a power of a umber by givig the followig explaatio: We assume without loss of geerality that is power of [your umber] This will ot iuece the al boud i ay importat way, after all, is at most a multiplicative factor away from some power of [your umber] 1 See the proof of Master Theorem i the book, page 49 1

2 Hadout: How to calculate time complexity? CSE 101 Witer 014 Evaluatig sums Arithmetic progressio Geometric progressio T = T 1 + c T = T 1 c We have: +c +c +c +c T 0 T 1 T 1 T T = rst + c (# of times c has bee added) rst + last T 0 + T T = (# of terms) Example: = 1 + c c c c T 0 T 1 T 1 T We have (c 6= 1): (# of times c has bee multiplied) T = rst c T 0 + T T = rst c(# of terms) 1 c 1 Example (c 6= 1): 1 + c + c + + c = c+1 1 c 1 Examples A) Fast multiplicatio Algorithm The idea is to square a to get a (oe multiplicatio) ad whe is odd, evaluate a 1 the same way ad the multiply it by a to get a (oe multiplicatio) Pseudo-code procedure mult(a, ): if == 1: retur a else if is eve: b = mult(a, / ) retur b * b else retur a * mult(a, - 1) Time complexity equatio Let T () be the umber of multiplicatios We assume without loss of geerality that is power of This will ot iuece the al boud i ay importat way, after all, is at most a multiplicative factor away from some power of At each step is divided by T () = T + 1

3 Hadout: How to calculate time complexity? CSE 101 Witer 014 Evaluatig T () Total work for step Total work for step Work at step (1 multiplicatio) The Master Theorem gives us a = 1 b = d = 0 so log b a = log 1 = 0 = d so T () = O(log ) B) Tower of Haoi: Lookig at the recursio tree Algorithm The algorithm requires that to move disks, we move rst 1 disks, the 1 ad ally 1 agai Time complexity equatio Let T () be the umber of moves T () = T ( 1) T ( 1) Total work for disks Total work for 1 disks Move 1 disk Total work for 1 disks We ca't apply Master Theorem o the complexity equatio T () = T ( evaluate T () to be able to give a big-o boud 1) + 1 so we have to The recursio tree Let's cosider the recurrece sub-problem tree: Level 0 : 1 problem Level 1 : problems 1 1 Level : 4 problems Level k: k Level 1 problems k k k k At the k-th level, there are k sub-problems that all require 1 move to get to level k 1 So, there are w(k) = k moves at level k (for level k oly) Google "tower of haoi recursive solutio" if you do't remember the solutio 3

4 Hadout: How to calculate time complexity? CSE 101 Witer 014 Summig to evaluate T () As T () is the sum of the moves of each level, we get: T () = We recogise here the sum of the rst terms of a geometric progressio whose ratio is ad rst term is 1 so we get: Therefore, T () = O( ) T () = 1 1 = 1 C) Tower of Haoi: Guessig with the first terms the prove Whe you have the recursio equatio T () = T ( by lookig at the rst terms: 1) + 1, you ca try to guess the value of T () T (0) = 0; T (1) = 1; T () = 3; T (3) = 7; T (4) = 15; T (5) = 31; T (6) = 63; : : : We ca guess it may be T () = 1, but we have to prove it! Let's do it by recursio o : Base case T (0) = 0 OK! Iductive hypothesis Let's suppose T () = 1 for a 0 Iductive step We have: T ( + 1) = T () + 1 = ( 1) + 1 (id hyp) = +1 1 Coclusio We have proved that T () = 1 for all 0 Therefore, T () = O( ) D) Tower of Haoi: Guessig by urollig the prove Aother way we ca guess the solutio is by urollig the recurrece, by substitutig it ito itself: T () = T ( 1) + 1 = ( T ( ) + 1) + 1 = 4 T ( ) + 3 = 4( T ( 3) + 1) + 3 = T ( 3) + 7 : : : Here we ca guess a ew recurrece: T () = k T ( k) + ( k 1) But we have also to prove it! Let's do it by recursio o k: 4

5 Hadout: How to calculate time complexity? CSE 101 Witer 014 Base case For k = 0, we have 0 T ( 0) + ( 0 1) = T () Ok! Iductive hypothesis Let's suppose T () = k T ( k) + ( k 1) for a k 0 Iductive step We have: T () = k T ( k) + ( k 1) = k ( T ( k 1) + 1) + ( k 1) itial recursio = k+1 T ( (k + 1)) + ( k+1 1) Coclusio We have proved that T () = k T ( k) + ( k 1) for k 0 Therefore, for k = we get T () = 1 so T () = O( ) E) Closest Pair of Poits i a Plae Let's evaluate the time complexity of the closest pair of poits i a plae solutio usig Divide ad Coquer is a power of We assume without loss of geerality that is power of This will ot iuece the al boud i ay importat way, after all, is at most a multiplicative factor away from some power of Sortig the poits before solvig First, before startig to solve the problem, we sort twice: Oce by the x-coordiate ad oce by the y-coordiate No further sortig is required at subsequet recursive steps Usig a stadard sortig algorithm requires O( log ), for example via mergesort Solvig For a set of poits i the plae (let T () be the time complexity): We partitio the curret set ito sub-sets deed by media x-coordiate As the poits have bee sorted by x-coordiate, dig the media requires oly a costat time operatio Recursively compute the closest pair distaces for the sub-sets T ( ) + T ( ) This requires a complexity of Deal with the middle ribbo Worst case sceario: all poits from the left side sub-problem ( poits) are i the ribbo ad for each of them, we have to check a maximum of 6 poits i the right side As the poits are sorted by y-coordiates ad we step through at most, checkig the middle ribbo takes at most a liear amout of time Fially take the miimum distace amog: the smallest of the right side, the smallest of the left side, ad the smallest of the middle ribbo That takes a costat time: O(1) 5

6 Hadout: How to calculate time complexity? CSE 101 Witer 014 We get: T () = O() + T + T + O() + O(1) Work for poits Splittig the plae (x-coord media) Work for the left side Work for the right side Work for the ribbo Combiig results (mi of 3 umbers) It ca be simplied i: Master Theorem gives us a = b = d = 1 T () = T + O() so log b a = log 1 = 1 = d so T () = O( log ) Coclusio We did steps: before solvig, we sorted the poits with a complexity of O( log ), ad we solved the problem with a complexity of T () = O( log ) So the overall time complexity is O( log ) F) Recursio tree agai We are give: T () = T + log We assume without loss of geerality that is power of This will ot iuece the al boud i ay importat way, after all, is at most a multiplicative factor away from some power of We ca't use the Master Theorem so we have to evaluate T () to be able to give the complexity i big-o terms Let's cosider the recurrece sub-problem tree: Level 0 : 1 problem Level 1 : problems = = Level : 4 problems =4 =4 =4 =4 Level k: k problems = k = k = k = k Level log

7 Hadout: How to calculate time complexity? CSE 101 Witer 014 As we kow the size of the sub-problems at level k is k accordig to the complexity equatio: The work for oe sub-problem of level k is, k log k = 1 k log() k As there is k sub-problems at level k, the total work made at level k is: k 1 k log() k = log() k # of problems at level k work for each problem of level k total work made at level k Therefore: T () = (work at level 0) + (work at level 1) + + (work at level log() - 1) = log() + log() = log() 1 + 1! log() We saw i the homework that we had: m = O(log m) I our previous sum, m = log so we get T () = O( log(log )) 7