Asymptotic Growth of Functions


 Milo Clarke
 1 years ago
 Views:
Transcription
1 CMPS Itroductio to Aalysis of Algorithms Fall 3 Asymptotic Growth of Fuctios We itroduce several types of asymptotic otatio which are used to compare the performace ad efficiecy of algorithms As we ll see, the asymptotic ru time of a algorithm gives a simple, ad machie idepedet, characterizatio of it s complexity Defiitio Let g ( be a fuctio The set O ( is defied as O = { f ( c >, >, : f ( c ) } ( I other words, f ( O( if ad oly if there exist positive costats c, ad, such that for all, the iequality f ( c is satisfied We say that f ( is Big O of g (, or that g ( is a asymptotic upper boud for f ( We ofte abuse otatio slightly by writig f ( to mea f ( O( Actually f ( O( is also a abuse of otatio We should really write f O(g) sice what we have defied is a set of fuctios, ot a set of umbers The otatioal covetio O ( is useful sice it allows us to refer to the set O ( 3 ) say, without havig to itroduce a fuctio symbol for the 3 polyomial Observe that if f ( the f ( is asymptotically oegative, ie f ( is oegative for all sufficietly large, ad liewise for g ( We mae the blaet assumptio from ow o that all fuctios uder discussio are asymptotically oegative I practice we will be cocered with iteger valued fuctios of a (positive) iteger ( g : Z + Z + ) However, i what follows, it is useful to cosider to be a cotiuous real variable taig positive values ad g to be real valued fuctio ( g : R + R + ) Geometrically f ( says: cg ( f (
2 Example ) Observe that for all, as ca be easily verified Thus we may tae = ad c = i the defiitio Note that i this example, ay value of greater tha will also wor, ad liewise ay value of c greater tha wors I geeral if there exist positive costats ad c such that f ( c for all, the ifiitely may such costats also exist I order to prove that f ( it is ot ecessary to fid the smallest possible ad c maig the f ( c true It is oly ecessary to show that at least oe pair of such costats exist Geeralizig the last example, we will show that a + b c + d + e) for ay costats ae, ad i fact p ( q( wheever p ( ad q ( are polyomials with de p) de q) Defiitio Let g ( be a fuctio ad defie the set Ω ( to be Ω = { f ( c >, >, : c f ( )} ( We say f ( is big Omega of g (, ad that g ( is a asymptotic lower boud for f ( As before we write f ( to mea f ( Ω( The geometric iterpretatio is: f ( cg (
3 Lemma f ( if ad oly if f ( Proof: If f ( the there exist positive umbers c, such that f ( c for all Let c = / c ad = The c f ( for all, provig f ( The coverse is similar ad we leave it to the reader /// Defiitio Let g ( be a fuctio ad defie the set Θ ( Ω( Equivaletly Θ( = { f ( c >, c >, >, : c f ( c } We write f ( ad say the g ( is a asymptotically tight boud for f (, or that f ( ad g ( are asymptotically equivalet We iterpret this geometrically as: c f ( c Exercise Prove that if c is a positive costat, the cf ( f ( Exercise Prove that f ( if ad oly if f ( Example Prove that ) + Proof: Accordig to the defiitio, we must fid positive umbers c, c,, such that the iequality c + c holds for all Pic c =, c =, ad = The if we have: ad ( ) ad ( ) ( c ) ad ( c ) + c ad + c, c + c, + c, c as required /// 3
4 The reader may fid our choice of values for the costats c, c, i this example somewhat mysterious Adequate values for these costats ca usually be obtaied by worig bacwards algebraically from the iequality to be proved Notice that i this example there are may valid choices For istace oe checs easily that c /, c 3/, ad wor equally well = Exercise Let a, b be real umbers with b > Prove directly from the defiitio (as above) that b b ( + a) ) (I what follows we lear a much easier way to prove this) Lemma If f ( h( for all sufficietly large, ad if h (, the f ( Proof: The above hypotheses say that there exist positive umbers c ad such that h( c for all Also there exists such that f ( h( for all (Recall f ( is assumed to be asymptotically oegative) Defie = max(, ), so that if we have both ad Thus implies f ( c, ad therefore f ( /// Exercise Prove that if h ( f ( h ( for all sufficietly large, where h ( ad h ( )), the f ( + Example Let be a fixed iteger Prove that i ) + + Proof: Observe that i = = ), ad i i / / ( / ) = = / ( / ) ( / )( / ) = (/ ) ) + By the result of the precedig exercise, we coclude i ) /// Whe asymptotic otatio appears i a formula such as T ( = T ( / ) + Θ( we iterpret Θ ( to 3 3 stad for some aoymous fuctio i the class Θ ( For example = 3 + Θ( ) Here Θ ( ) stads for 4 +, which belogs to the class Θ ( ) The expressio = Θ( i) ca be puzzlig O the surface it stads for Θ( ) + Θ() + Θ(3) + + Θ(, i which is meaigless sice Θ (costat) cosists of all fuctios which are bouded above by some costat We iterpret Θ (i) i this expressio to stad for a sigle fuctio f (i) i the class Θ (i), evaluated at i =,, 3,, Exercise Prove that = Θ( i) ) The left had side stads for a sigle fuctio f (i) summed i = ( i for i =,, 3,, By the previous exercise it is sufficiet to show that h f ( i) h ( ) for all sufficietly large, where h ( ) ad h ( ) 4
5 Defiitio o ( = { f ( c >, >, : f ( < c } We say that g ( is a strict Asymptotic upper boud for f ( ad write f ( as before f ( Lemma f ( if ad oly if lim = f ( Proof: Observe that f ( if ad oly if c >, >, : < c, which is the f ( very defiitio of the limit statemet lim = /// l Example l ) sice lim = (Apply l Hopitals rule) Example b ) for ay > ad b > sice lim = b other words, ay expoetial grows strictly faster tha ay polyomial (Apply l Hopitals rule times) I By comparig defiitios of o ( ad O( oe sees immediately that o( O( Also o fuctio ca belog to both o ( ad Ω (, as is easily verified (exercise) Thus o ( Ω( =, ad therefore o( O( Defiitio ω ( g ( = { f ( c >, >, : c < f ( } Here we say that g ( is a strict asymptotic lower boud for f ( ad write f ( = ω( f ( Exercise Prove that f ( = ω( if ad oly if lim = g ( ) Exercise Prove ω ( O( =, whece ω ( Ω( The followig picture emerges: O ( Ω ( o ( Θ ( ω ( 5
6 Lemma If f ( lim = L, where L <, the f ( Proof: The defiitio of the above limit is ε =, there exists a positive umber such that for all : f ( L < f ( < L < f ( < L + f ( < ( L + ) f ( ε >, >, : L < ε Thus if we let Now tae c = L + i the defiitio of O, so that f ( as claimed /// f ( Lemma If lim = L, where < L, the f ( Proof: The limit statemet implies lim = L, where L =/ L ad hece L < By the f ( previous lemma g ( f (, ad therefore f ( /// Exercise Prove that if f ( lim = L, where < L <, the f ( Although o (, ω (, ad a certai subset of Θ ( are characterized by limits, the full sets O (, Ω (, ad Θ ( have o such characterizatio as the followig examples show Example A Let g ( = ad f ( = ( + si( g ( f ( f ( Clearly f (, but = + si(, whose limit does ot exist This example shows that the cotaimet o( O( is i geeral strict sice f ( Ω( (exercise) Therefore f ( Θ(, so that f ( O( But f ( o( sice the limit does ot exist 6
7 Example B Let g ( = ad f ( = ( + si( 3g ( f ( g ( Sice ( + si( 3 for all, we have f (, but f ( = + si( whose limit does ot exist Exercise Fid fuctios f ( ad g ( such that f ( Ω(, but exist (eve i the sese of beig ifiite), so that f ( ω( f ( lim g ( ) does ot The precedig limit theorems ad couterexamples ca be summarized i the followig diagram f ( Here L deotes the limit L = lim, if it exists g ( ) O ( Ω ( Θ ( o ( ω ( L = < L < L = Ex A Ex B I spite of the above couterexamples, the precedig limit theorems are a very useful tool for establishig asymptotic comparisos betwee fuctios For istace recall the earlier exercise to b b show ( + a) ) for real umbers a, ad b with b > The result follows immediately from sice << b ( + a) lim = lim = = + a b, b 7 b
8 Exercise Use limits to prove the followig: a l ) (here l ) deotes the base logarithm of ) 5 b = ω( ) c If P ( is a polyomial of degree, the P( ) d f ( + o( f ( f ( (Oe ca always disregard lower order terms) ε e (log ) for ay > ad ε > (Polyomials grow faster tha logs) ε f b ) for ay ε > ad b > (Expoetials grow faster tha polyomials) There is a aalogy betwee the asymptotic compariso of fuctios f ( ad g (, ad the compariso of real umbers x ad y f ( ~ x y f ( ~ x = y f ( ~ x y f ( ~ x < y f ( = ω( ~ x > y Note however that this aalogy is ot exact sice there exist pairs of fuctios which are ot comparable, while ay two real umbers are comparable (See problem 3c, p58) 8
SOME GEOMETRY IN HIGHDIMENSIONAL SPACES
SOME GEOMETRY IN HIGHDIMENSIONAL SPACES MATH 57A. Itroductio Our geometric ituitio is derived from threedimesioal space. Three coordiates suffice. May objects of iterest i aalysis, however, require far
More informationHOW MANY TIMES SHOULD YOU SHUFFLE A DECK OF CARDS? 1
1 HOW MANY TIMES SHOULD YOU SHUFFLE A DECK OF CARDS? 1 Brad Ma Departmet of Mathematics Harvard Uiversity ABSTRACT I this paper a mathematical model of card shufflig is costructed, ad used to determie
More informationConsistency of Random Forests and Other Averaging Classifiers
Joural of Machie Learig Research 9 (2008) 20152033 Submitted 1/08; Revised 5/08; Published 9/08 Cosistecy of Radom Forests ad Other Averagig Classifiers Gérard Biau LSTA & LPMA Uiversité Pierre et Marie
More information4. Trees. 4.1 Basics. Definition: A graph having no cycles is said to be acyclic. A forest is an acyclic graph.
4. Trees Oe of the importat classes of graphs is the trees. The importace of trees is evidet from their applicatios i various areas, especially theoretical computer sciece ad molecular evolutio. 4.1 Basics
More informationStéphane Boucheron 1, Olivier Bousquet 2 and Gábor Lugosi 3
ESAIM: Probability ad Statistics URL: http://wwwemathfr/ps/ Will be set by the publisher THEORY OF CLASSIFICATION: A SURVEY OF SOME RECENT ADVANCES Stéphae Bouchero 1, Olivier Bousquet 2 ad Gábor Lugosi
More informationWhich Extreme Values Are Really Extreme?
Which Extreme Values Are Really Extreme? JESÚS GONZALO Uiversidad Carlos III de Madrid JOSÉ OLMO Uiversidad Carlos III de Madrid abstract We defie the extreme values of ay radom sample of size from a distributio
More informationON THE EVOLUTION OF RANDOM GRAPHS by P. ERDŐS and A. RÉNYI. Introduction
ON THE EVOLUTION OF RANDOM GRAPHS by P. ERDŐS ad A. RÉNYI Itroductio Dedicated to Professor P. Turá at his 50th birthday. Our aim is to study the probable structure of a radom graph r N which has give
More informationBOUNDED GAPS BETWEEN PRIMES
BOUNDED GAPS BETWEEN PRIMES ANDREW GRANVILLE Abstract. Recetly, Yitag Zhag proved the existece of a fiite boud B such that there are ifiitely may pairs p, p of cosecutive primes for which p p B. This ca
More informationNo Eigenvalues Outside the Support of the Limiting Spectral Distribution of Large Dimensional Sample Covariance Matrices
No igevalues Outside the Support of the Limitig Spectral Distributio of Large Dimesioal Sample Covariace Matrices By Z.D. Bai ad Jack W. Silverstei 2 Natioal Uiversity of Sigapore ad North Carolia State
More informationEverything You Always Wanted to Know about Copula Modeling but Were Afraid to Ask
Everythig You Always Wated to Kow about Copula Modelig but Were Afraid to Ask Christia Geest ad AeCatherie Favre 2 Abstract: This paper presets a itroductio to iferece for copula models, based o rak methods.
More informationTesting for Welfare Comparisons when Populations Differ in Size
Cahier de recherche/workig Paper 039 Testig for Welfare Comparisos whe Populatios Differ i Size JeaYves Duclos Agès Zabsoré Septembre/September 200 Duclos: Départemet d écoomique, PEP ad CIRPÉE, Uiversité
More informationSystemic Risk and Stability in Financial Networks
America Ecoomic Review 2015, 105(2): 564 608 http://dx.doi.org/10.1257/aer.20130456 Systemic Risk ad Stability i Fiacial Networks By Daro Acemoglu, Asuma Ozdaglar, ad Alireza TahbazSalehi * This paper
More informationHow Has the Literature on Gini s Index Evolved in the Past 80 Years?
How Has the Literature o Gii s Idex Evolved i the Past 80 Years? Kua Xu Departmet of Ecoomics Dalhousie Uiversity Halifax, Nova Scotia Caada B3H 3J5 Jauary 2004 The author started this survey paper whe
More informationCrowds: Anonymity for Web Transactions
Crowds: Aoymity for Web Trasactios Michael K. Reiter ad Aviel D. Rubi AT&T Labs Research I this paper we itroduce a system called Crowds for protectig users aoymity o the worldwideweb. Crowds, amed for
More informationWhich Codes Have CycleFree Tanner Graphs?
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 45, NO. 6, SEPTEMBER 1999 173 Which Coes Have CycleFree Taer Graphs? Tuvi Etzio, Seior Member, IEEE, Ari Trachteberg, Stuet Member, IEEE, a Alexaer Vary,
More informationType Less, Find More: Fast Autocompletion Search with a Succinct Index
Type Less, Fid More: Fast Autocompletio Search with a Succict Idex Holger Bast MaxPlackIstitut für Iformatik Saarbrücke, Germay bast@mpiif.mpg.de Igmar Weber MaxPlackIstitut für Iformatik Saarbrücke,
More informationSignal Reconstruction from Noisy Random Projections
Sigal Recostructio from Noisy Radom Projectios Jarvis Haut ad Robert Nowak Deartmet of Electrical ad Comuter Egieerig Uiversity of WiscosiMadiso March, 005; Revised February, 006 Abstract Recet results
More informationThe Review of Economic Studies Ltd.
The Review of Ecoomic Studies Ltd. Walras' Tâtoemet i the Theory of Exchage Author(s): H. Uzawa Source: The Review of Ecoomic Studies, Vol. 27, No. 3 (Ju., 1960), pp. 182194 Published by: The Review of
More informationThe Arithmetic of Investment Expenses
Fiacial Aalysts Joural Volume 69 Number 2 2013 CFA Istitute The Arithmetic of Ivestmet Expeses William F. Sharpe Recet regulatory chages have brought a reewed focus o the impact of ivestmet expeses o ivestors
More informationA Kernel TwoSample Test
Joural of Machie Learig Research 3 0) 73773 Subitted 4/08; Revised /; Published 3/ Arthur Gretto MPI for Itelliget Systes Speastrasse 38 7076 Tübige, Geray A Kerel TwoSaple Test Karste M. Borgwardt Machie
More informationRevised Version of Chapter 23. We learned long ago how to solve linear congruences. ax c (mod m)
Chapter 23 Squares Modulo p Revised Version of Chapter 23 We learned long ago how to solve linear congruences ax c (mod m) (see Chapter 8). It s now time to take the plunge and move on to quadratic equations.
More informationON AN INTEGRAL OPERATOR WHICH PRESERVE THE UNIVALENCE
Proceedigs of the Iteratioal Coferece o Theory ad Applicatios of Mathematics ad Iformatics ICTAMI 3, Alba Iulia ON AN INTEGRAL OPERATOR WHICH PRESERVE THE UNIVALENCE by Maria E Gageoea ad Silvia Moldoveau
More informationTeaching Bayesian Reasoning in Less Than Two Hours
Joural of Experimetal Psychology: Geeral 21, Vol., No. 3, 4 Copyright 21 by the America Psychological Associatio, Ic. 963445/1/S5. DOI: 1.7//963445..3. Teachig Bayesia Reasoig i Less Tha Two Hours Peter
More informationThe Unicorn, The Normal Curve, and Other Improbable Creatures
Psychological Bulleti 1989, Vol. 105. No.1, 156166 The Uicor, The Normal Curve, ad Other Improbable Creatures Theodore Micceri 1 Departmet of Educatioal Leadership Uiversity of South Florida A ivestigatio
More informationController Area Network (CAN) Schedulability Analysis with FIFO queues
Cotroller Area Network (CAN) Schedulability Aalysis with FIFO queues Robert I. Davis RealTie Systes Research Group, Departet of Coputer Sciece, Uiversity of York, YO10 5DD, York, UK rob.davis@cs.york.ac.uk
More informationSolutions to Homework 6 Mathematics 503 Foundations of Mathematics Spring 2014
Solutions to Homework 6 Mathematics 503 Foundations of Mathematics Spring 2014 3.4: 1. If m is any integer, then m(m + 1) = m 2 + m is the product of m and its successor. That it to say, m 2 + m is the
More informationWHAT ARE MATHEMATICAL PROOFS AND WHY THEY ARE IMPORTANT?
WHAT ARE MATHEMATICAL PROOFS AND WHY THEY ARE IMPORTANT? introduction Many students seem to have trouble with the notion of a mathematical proof. People that come to a course like Math 216, who certainly
More informationJ. J. Kennedy, 1 N. A. Rayner, 1 R. O. Smith, 2 D. E. Parker, 1 and M. Saunby 1. 1. Introduction
Reassessig biases ad other ucertaities i seasurface temperature observatios measured i situ sice 85, part : measuremet ad samplig ucertaities J. J. Keedy, N. A. Rayer, R. O. Smith, D. E. Parker, ad M.
More informationCatalogue no. 62557XPB Your Guide to the Consumer Price Index
Catalogue o. 62557XPB Your Guide to the Cosumer Price Idex (Texte fraçais au verso) Statistics Caada Statistique Caada Data i may forms Statistics Caada dissemiates data i a variety of forms. I additio
More informationDryad: Distributed DataParallel Programs from Sequential Building Blocks
Dryad: Distributed DataParallel Programs from Sequetial uildig locks Michael Isard Microsoft esearch, Silico Valley drew irrell Microsoft esearch, Silico Valley Mihai udiu Microsoft esearch, Silico Valley
More information