Lecture 3. denote the orthogonal complement of S k. Then. 1 x S k. n. 2 x T Ax = ( ) λ x. with x = 1, we have. i = λ k x 2 = λ k.


 Charla Wilcox
 2 years ago
 Views:
Transcription
1 A Algorithmist s Toolkit September 17, 009 Lecture 3 Lecturer: Joatha Keler Scribe: Adre Wibisoo 1 Outlie Today s lecture covers three mai parts: CouratFischer formula ad Rayleigh quotiets The coectio of λ to graph cuttig Cheeger s Iequality CouratFischer ad Rayleigh Quotiets The CouratFischer theorem gives a variatioal formulatio of the eigevalues of a symmetric matrix, which ca be useful for obtaiig bouds o the eigevalues. Theorem 1 (CouratFischer Formula) Let A be a symmetric matrix with eigevalues λ 1 λ... λ ad correspodig eigevectors v 1,...,v. The λ 1 = mi = mi, x =1 x 0 x T x λ = mi = mi, x =1 x 0 x T x xv 1 xv 1 λ = λ max = max = max. x =1 x 0 x T x I geeral, for 1 k, let S k deote the spa of v 1,...,v k (with S 0 = {0}), ad let S k deote the orthogoal complemet of S k. The λ k = mi = mi. x =1 x 0 x T x x S k 1 x S k 1 Proof Let A = Q T ΛQ be the eigedecompositio of A. We observe that = x T Q T ΛQx = (Qx) T Λ(Qx), ad sice Q is orthogoal, Qx = x. Thus it suffices to cosider the case whe A = Λ is a diagoal matrix with the eigevalues λ 1,...,λ i the diagoal. The we ca write λ 1 x 1. = ( ) x 1 x... = λ i x i. λ x i=1 We ote that whe A is diagoal, the eigevectors of A are v k = e k, the stadard basis vector i R, i.e. (e k ) i =1 if i = k, ad (e k ) i = 0 otherwise. The the coditio x S k 1 implies x e i for i =1,...,k 1, so x i = x, e i = 0. Therefore, for x S k 1 with x = 1, we have = λ i x i = λ i x i λ k x i = λ k x = λ k. i=1 i=k i=k 31
2 O the other had, pluggig i x = e k S k 1 yields = (e k ) T Ae k = λ k. This shows that Similarly, for x = 1, λ k = mi. x =1 x S k 1 = λ i x i λ max x i = λ max x = λ max. i=1 i=1 O the other had, takig x = e yields = (e ) T Ae = λ max. Hece we coclude that λ max = max. x =1 The Rayleigh quotiet is the applicatio of the CouratFischer Formula to the Laplacia of a graph. Corollary (Rayleigh Quotiet) Let G = (V, E) be a graph ad L be the Laplacia of G. We already kow that the smallest eigevalue is λ 1 =0 with eigevector v 1 = 1. By the CouratFischer Formula, (i,j) E (x i x j ) λ = mi = mi, x =0 x T x x=0 xv 1 x1 i V x i (i,j) E (x i x j ) λ max = max = max. x =0 x T x x=0 i V x i We ca iterpret the formula for λ as puttig sprigs o each edge (with slightly weird boudary coditios correspodig to ormalizatio) ad miimizig the potetial eergy of the cofiguratio. Some big matrices are hard or aoyig to diagoalize, so i some cases, we may ot wat to calculate the exact value of λ. However, we ca still get a approximatio by just costructig a vector x that has a small Rayleigh quotiet. Similarly, we ca fid a lower boud o λ max by costructig a vector that has a large Rayleigh quotiet. We will look at two examples i which we boud λ..1 Example 1: The Path Graph Let P +1 be the path graph of +1 vertices. Label the vertices as 0, 1,..., from oe ed of the path to the other. Cosider the vector x R +1 give by x i =i for vertices i = 0, 1,...,. Note that i=0 x i =0, so x 1. Calculatig the Rayleigh quotiet for x gives us ( ) (i,j) E (x i x j ) = = = O. i V x i i=0(i ) Ω( 3 ) Thus we ca boud λ O(1/ ). We kew this was true from the explicit formula of λ i terms of sies ad cosies from Lecture, but this is much cleaer ad more geeral of a result.. Example : A Complete Biary Tree Let G be a complete biary tree o = h 1 odes. Defie the vector x R to have the value 0 o the root ode, 1 o all odes i the left subtree of the root, ad 1 o all odes i the right subtree of the root. 3
3 It is easy to see that i V x i = 0, sice there are equal umbers of odes o the left ad right subtrees of the root, so x 1. Calculatig the Rayleigh quotiet of x gives us ( ) (i,j) E (x i x j ) 1 = = O. x 1 i V i Thus we get λ O(1/), agai with little effort. It turs out i this case that our approximatio is correct withi a costat factor, ad we did ot eve eed to diagoalize a big matrix. 3 Graph Cuttig The basic problem of graph cuttig is to cut a give graph G ito two pieces such that both are pretty big. Graph cuttig has may applicatios i computer sciece ad computig, e.g. for parallel processig, divideadcoquer algorithms, or clusterig. I each applicatio, we wat to divide the problem ito smaller pieces so as to optimize some measure of efficiecy, depedig o the specific problems. 3.1 How Do We Cut Graphs? The first questio to ask about graph cuttig is what we wat to optimize whe we are cuttig a graph. Before attemptig to aswer this questio, we itroduce several otatios. Let G = (V, E) be a graph. Give a set S V of vertices of G, let S = V \ S be the complemet of S i V. Let S ad S deote the umber of vertices i S ad S, respectively. Fially, let e(s) deote the umber of edges betwee S ad S. Note that e(s) = e(s ). Now we cosider some possible aswers to our earlier questio. Attempt 1: Micut. Divide the vertex set V ito two parts S ad S to miimize e(s). This approach is motivated by the ituitio that to get a good cut, we do ot wat to break too may edges. However, this approach aloe is ot sufficiet, as Figure 1(a) demostrates. I this example, we ideally wat to cut the graph across the two edges i the middle, but the micut criterio would result i a cut across the oe edge o the right. Attempt : Approximate bisectio. Divide the vertex set V ito two parts S ad S, such that S ad S are approximately / (or at least /3). This criterio would take care of the problem metioed i Figure 1(a), but it is also ot free of problems, as Figure 1(b) shows. I this example, we ideally wat to cut the graph across the oe edge i the middle that separates the two clusters. However, the approximate bisectio criterio would force us to make a cut across the dese graph o the left. (a) Problem with micut (b) Problem with approximate bisectio Figure 1: Illustratio for problems with the proposed graph cuttig criteria. Now we propose a criterio for graph cuttig that balaces the two approaches above. Defiitio 3 (Cut Ratio) The cut ratio φ of a cut S S is give by e(s) φ(s) = mi( S, S ). 33
4 The cut of miimum ratio is the cut that miimizes φ(s). The isoperimetric umber of a graph G is the value of the miimum cut, φ(g) = mi φ(s). S V As we ca see from the defiitio above, the cut ratio is tryig to miimize the umber of edges across the cut, while pealizig cuts with small umber of vertices. This criterio turs out to be a good oe, ad is widely used for graph cuttig i practice. 3. A Iteger Program for the Cut Ratio Now that we have a good defiitio of graph cuttig, the questio is how to fid the optimal cut i a reasoable time. It turs out that we ca cast the problem of fidig cut of miimum ratio as a iteger program as follows. Associate every cut S S with a vector x { 1, 1}, where { 1, if i S, ad x i = 1, if i S. The it is easy to see that we ca write 1 e(s) = (x i x j ). 4 (i,j) E For a boolea statemet A, let [A] deote the characteristic fuctio o A, so[a] =1if A is true, ad [A] =0 if A is false. The we also have ( ) S S = [i S] [j S] = [i S, j S]= 1 1 [x i = x j ]= (x i x j ). 4 i V j V i,j V i,j V i<j Combiig the two computatios above, (i,j) E (x i x j ) e(s) mi = mi. x { 1,1} (x i x j ) S V S S Now ote that if V = S + S =, the Therefore, solvig the iteger program (i,j) E(x i x j ) mi x { 1,1} (x i x j ) i<j mi( S, S ) S S mi( S, S ), so we get 1 e(s) (i,j) E (x i x j ) e(s) φ(g) = mi mi mi = φ(g). S V mi( S, S ) x { 1,1} (x i x j ) S V mi( S, S ) i<j i<j allows us to approximate φ(g) withi a factor of. The bad ews is that it is NPhard to solve this program. However, if we remove the x { 1, 1} costrait, we ca actually solve the program. Note that removig the costrait x { 1, 1} is actually the same as sayig that x [ 1, 1], sice we ca scale x without chagig the value of the objective fuctio. 34
5 3.3 Iterlude o Relaxatios The idea to drop the costrait x { 1, 1} metioed i the previous sectio is actually a recurrig techique i algorithms, so it is worthwhile to give a more geeral explaatio of this relaxatio techique. A commo setup i approximatio algorithms is as follows: we wat to solve a NPhard questio which takes the form of miimizig f(x) subject to the costrait x C. Istead, we miimize f(x) subject to a weaker costrait x C C (see Figure for a illustratio). Let p ad q be the poits that miimize f i C ad C, respectively. Sice C C, we kow that f(q) f(p). smaller f(x) C C p q q Figure : Illustratio of the relaxatio techique for approximatio algorithms. For this relaxatio to be useful, we have to show how to roud q to a feasible poit q C, ad prove f(q ) γf(q) for some costat γ 1. This implies f(q ) γf(q) γf(p), so this process gives us a γapproximatio. 3.4 Solvig the Relaxed Program Goig back to our iteger program to fid the cut of miimum ratio, ow cosider the followig relaxed program, (i,j) E(x i x j ) mi. x R i<j (x i x j ) Sice the value of the objective fuctio oly depeds o the differeces x i x j, we ca traslate x R such that x 1, i.e. i=1 x i =0. The observe that (x i x j ) = x i, i<j i=1 which ca be obtaied either by expadig the summatio directly, or by otig that x is a eigevector of the Laplacia of the complete graph K with eigevalue (as we saw i Lecture ). Therefore, usig the Rayleigh quotiet, (i,j) E (x i x j ) (i,j) E (x i x j ) λ mi = mi =. x R i<j (x i x j ) x R i=1 x i x1 35
6 Puttig all the pieces together, we get 4 Cheeger s Iequality e(s) φ(g) = mi S V mi( S, S ) e(s) mi S V S S (i,j) E (x i x j ) = mi x { 1,1} i<j (x i x j ) (i,j) E (x i x j ) mi x R i<j (x i x j ) (i,j) E (x i x j ) = mi x R i=1 x i x1 λ =. I the previous sectio, we obtaied the boud φ(g) λ /, but what about the other directio? For that, we would eed a roudig method, which is a way of gettig a cut from λ ad v, ad a upper boud o how much ithe roudig icreases the cut ratio that we are tryig to miimize. I the ext sectio, we will see how to costruct a cut from λ ad v that gives us the followig boud, which is Cheeger s Iequality. Theorem 4 (Cheeger s Iequality) Give a graph G, where d max is the maximum degree i G. φ(g) λ φ(g), d max As a side ote, the d max disappears from the formula if we use the ormalized Laplacia i our calculatios, but the proof is messier ad is ot fudametally ay differet from the proof usig the regular Laplacia. The lower boud of φ(g) /d max i Cheeger s Iequality is the best we ca do to boud λ. The square factor φ(g) is ufortuate, but if it were withi a costat factor of φ(g), we would be able to fid a costat approximatio of a NPhard problem. Also, if we look at the examples of the path graph ad the complete biary tree, their isoperimetric umbers are the same sice we ca cut exactly oe edge i the middle of the graph ad divide the graphs ito two asymptotically equalsized pieces for a value of O(1/). However, the two graphs have differet upper bouds for λ, O(1/ ) ad O(1/) respectively, which demostrate that both the lower ad upper bouds of λ i Cheeger s iequality are tight (to a costat factor). 4.1 How to Get a Cut from v ad λ Let x R such that x 1. We will use x as a map from the vertices V to R. Cuttig R would thus give a partitio of V as follows: order the vertices such that x 1 x... x, ad the cut will be defied by the set S = {1,...,k} for some value of k. The value of k caot be kow a priori sice the best cut depeds o the graph. I practice, a algorithm would have to try all values of k to actually fid the optimal cut after embeddig the graph to the real lie. We will actually prove somethig slightly stroger tha Cheeger s Iequality: 36
7 Theorem 5 For ay x 1, x 1 x... x, there is some i for which x T Lx φ({1,...,i}). x T x d max This is great because it ot oly implies Cheeger s iequality by takig x = v, but it also gives a actual cut. It also works eve if we have ot calculated the exact values for λ ad v ; we just have to get a good approximatio of v ad we ca still get a cut. 4. Proof of Cheeger s Iequality 4..1 Step 1: Preprocessig First, we are goig to do some preprocessig. This step does ot reduce the geerality of the proof much, but it will make the actual proof cleaer. For simplicity, suppose is odd. Let m = ( +1)/. Defie the vector y by y i = x i x m. We ca observe that y m = 0, half of the vertices are to the left of y m, ad the other half are to the right of y m. Claim 6 Proof x T Lx y T Ly x T x y T y First, the umerators are equal by the operatio of the Laplacia, x T Lx = (x i x j ) ( = (yi + x m ) (y j + x m ) ) = (yi y j ) = y T Ly. (i,j) E (i,j) E (i,j) E Next, sice x 1, y T y = (x + x m 1) T (x + x m 1)= x T x +x m (x T 1)+ x m(1 T 1)= x T x + x m x T x. Puttig together the two computatios above yields the desired iequality. 4.. Step : A Little More Preprocessig We do ot wat edges crossig y m = 0 (because we will later cosider the positive ad egative vertices separately), so we replace ay edge (i, j) with two edges (i, m) ad (m, j). Call this ew edge set E. Claim 7 (i,j) E (y i y j ) (i,j) E (y i y j ). i V y i i V y i Proof The oly differece i the umerator comes from the edges (i, j) that we split ito (i, m) ad (m, j). I that case, it is easy to see that (also otig that y m =0) (y j y i ) (y j y m ) +(y m y i ). 37
8 4..3 Step 3: Breakig the Sum i Half We would like to break the summatios i half so that we do ot have to deal with separate cases with positive ad egative umbers. Let E be the edges (i, j) with i, j m, ad let E + be the edges (i, j) with i, j m. We the have (y i y j ) + (y i y j ) (i,j) E (y i y j ) (i,j) E (i,j) E+ = m. i y i i=1 y i + i=m yi Note that y m appears twice i the summatio o the deomiator, which is fie sice y m = 0. We also kow that for ay a, b, c, d > 0, ( ) a + b a b mi,, c + d c d so it is eough to boud (i,j) E (y i y j ) (y i y j ) (i,j) E+ m ad. i=1 y i i=m y i Sice the two values are essetially the same, we will focus oly o the first oe The Mai Lemma Let C i be the umber of edges crossig the poit x i, i.e. the umber of edges i the cut if we were to take S = {1,...,i}. Recall that e(s) φ = φ(g) = mi, S V mi( S, S ) so by takig S = {1,...,i}, weget C i φi for i / ad C i φ( i) for i /. The mai lemma we use to prove Cheeger s Iequality is as follows. Lemma 8 (Summatio by Parts) For ay z 1... z m =0, m z i z j φ z i. j 1 z i z j = z j z i =(z i+1 z i )+(z i+ z i+1 )+ +(z j z j 1 )= (z k+1 z k ). m 1 m 1 z i z j = C k (z k+1 z k ) φ k(z k+1 z k ). m 1 z i z j φ k(z k+1 z k ) ( ) = φ (z z 1 )+(z 3 z )+3(z 4 z 3 )+ +(m 1)(z m z m 1 ) m = φ z i. Proof For each (i, j) E with i <j, write Summig over (i, j) E, we observe that each term z k+1 z k appears exactly C k times. Therefore, i=1 (i,j) E k=1 k=1 Note that z i z m =0, so z i = z i for 1 i m. The we ca evaluate the last summatio above as (i,j) E k=1 = φ( z 1 z z m 1 +(m 1)z m ) i=1 k=i 38
9 4..5 Usig the Mai Lemma to Prove Cheeger s Iequality Now we ca fially prove Cheeger s iequality. Proof of Cheeger s Iequality: This proof has five mai steps. 1. First, we ormalize y such that m i=1 y i =1.. Next, this is perhaps a somewhat oituitive step, but we wat to get squares ito our expressio, so we apply the mai lemma (Lemma 8) to a ew vector z with z i = y i. We ow have m y i y j φ yi = φ. 3. Next, we wat somethig that looks like (y i y j ) istead of y i y j, so we are goig to use the CauchySchwarz iequality. 1/ 1/ y i y j = y i y j y i + y j (y i y j ) (y i + y j ). i=1 4. We wat to get rid of the (y i + y j ) part, so we boud it ad observe that the maximum umber of times ay y i ca show up i the summatio over the edges is the the maximum degree of ay vertex. m (y i + y j ) (y i + y j ) d max y i d max. 5. Puttig it all together, we get ( ) (y i y j ) (i,j) E yi y j φ m. (y i + y j ) d max (i,j) E Similarly, we ca also show that Therefore, i=1 y i (i,j) E (i,j) E + i=m y i d max { } T x Lx y T Ly (i,j) E (y i y j ) (y i y j ) φ (i,j) E mi m, +. x T x y T y y i=1 (y i y j ) φ. i=1 y i i=m i d max 4..6 So who is Cheeger ayway? Jeff Cheeger is a differetial geometer. His iequality makes a lot more sese i the cotiuous world, ad his motivatio was i differetial geometry. This was part of his PhD thesis, ad he was actually ivestigatig heat kerels o smooth maifolds. A heat kerel ca also be thought of as a poit of heat i space, ad the questio is the speed at which the heat spreads. It ca also be thought of as the mixig time of a radom walk, which will be discussed i future lectures. 39
10 MIT OpeCourseWare Topics i Theoretical Computer Sciece: A Algorithmist's Toolkit Fall 009 For iformatio about citig these materials or our Terms of Use, visit:
Lecture 4: Cheeger s Inequality
Spectral Graph Theory ad Applicatios WS 0/0 Lecture 4: Cheeger s Iequality Lecturer: Thomas Sauerwald & He Su Statemet of Cheeger s Iequality I this lecture we assume for simplicity that G is a dregular
More informationLecture 13. Lecturer: Jonathan Kelner Scribe: Jonathan Pines (2009)
18.409 A Algorithmist s Toolkit October 27, 2009 Lecture 13 Lecturer: Joatha Keler Scribe: Joatha Pies (2009) 1 Outlie Last time, we proved the BruMikowski iequality for boxes. Today we ll go over the
More informationRiemann Sums y = f (x)
Riema Sums Recall that we have previously discussed the area problem I its simplest form we ca state it this way: The Area Problem Let f be a cotiuous, oegative fuctio o the closed iterval [a, b] Fid
More informationProperties of MLE: consistency, asymptotic normality. Fisher information.
Lecture 3 Properties of MLE: cosistecy, asymptotic ormality. Fisher iformatio. I this sectio we will try to uderstad why MLEs are good. Let us recall two facts from probability that we be used ofte throughout
More informationSection IV.5: Recurrence Relations from Algorithms
Sectio IV.5: Recurrece Relatios from Algorithms Give a recursive algorithm with iput size, we wish to fid a Θ (best big O) estimate for its ru time T() either by obtaiig a explicit formula for T() or by
More information4.1 Sigma Notation and Riemann Sums
0 the itegral. Sigma Notatio ad Riema Sums Oe strategy for calculatig the area of a regio is to cut the regio ito simple shapes, calculate the area of each simple shape, ad the add these smaller areas
More informationI. Chisquared Distributions
1 M 358K Supplemet to Chapter 23: CHISQUARED DISTRIBUTIONS, TDISTRIBUTIONS, AND DEGREES OF FREEDOM To uderstad tdistributios, we first eed to look at aother family of distributios, the chisquared distributios.
More informationCME 302: NUMERICAL LINEAR ALGEBRA FALL 2005/06 LECTURE 8
CME 30: NUMERICAL LINEAR ALGEBRA FALL 005/06 LECTURE 8 GENE H GOLUB 1 Positive Defiite Matrices A matrix A is positive defiite if x Ax > 0 for all ozero x A positive defiite matrix has real ad positive
More informationAsymptotic Growth of Functions
CMPS Itroductio to Aalysis of Algorithms Fall 3 Asymptotic Growth of Fuctios We itroduce several types of asymptotic otatio which are used to compare the performace ad efficiecy of algorithms As we ll
More informationIn nite Sequences. Dr. Philippe B. Laval Kennesaw State University. October 9, 2008
I ite Sequeces Dr. Philippe B. Laval Keesaw State Uiversity October 9, 2008 Abstract This had out is a itroductio to i ite sequeces. mai de itios ad presets some elemetary results. It gives the I ite Sequeces
More informationDivide and Conquer, Solving Recurrences, Integer Multiplication Scribe: Juliana Cook (2015), V. Williams Date: April 6, 2016
CS 6, Lecture 3 Divide ad Coquer, Solvig Recurreces, Iteger Multiplicatio Scribe: Juliaa Cook (05, V Williams Date: April 6, 06 Itroductio Today we will cotiue to talk about divide ad coquer, ad go ito
More informationConvexity, Inequalities, and Norms
Covexity, Iequalities, ad Norms Covex Fuctios You are probably familiar with the otio of cocavity of fuctios. Give a twicedifferetiable fuctio ϕ: R R, We say that ϕ is covex (or cocave up) if ϕ (x) 0 for
More informationwhen n = 1, 2, 3, 4, 5, 6, This list represents the amount of dollars you have after n days. Note: The use of is read as and so on.
Geometric eries Before we defie what is meat by a series, we eed to itroduce a related topic, that of sequeces. Formally, a sequece is a fuctio that computes a ordered list. uppose that o day 1, you have
More informationApproximating Area under a curve with rectangles. To find the area under a curve we approximate the area using rectangles and then use limits to find
1.8 Approximatig Area uder a curve with rectagles 1.6 To fid the area uder a curve we approximate the area usig rectagles ad the use limits to fid 1.4 the area. Example 1 Suppose we wat to estimate 1.
More information5 Boolean Decision Trees (February 11)
5 Boolea Decisio Trees (February 11) 5.1 Graph Coectivity Suppose we are give a udirected graph G, represeted as a boolea adjacecy matrix = (a ij ), where a ij = 1 if ad oly if vertices i ad j are coected
More informationThe second difference is the sequence of differences of the first difference sequence, 2
Differece Equatios I differetial equatios, you look for a fuctio that satisfies ad equatio ivolvig derivatives. I differece equatios, istead of a fuctio of a cotiuous variable (such as time), we look for
More informationSequences II. Chapter 3. 3.1 Convergent Sequences
Chapter 3 Sequeces II 3. Coverget Sequeces Plot a graph of the sequece a ) = 2, 3 2, 4 3, 5 + 4,...,,... To what limit do you thik this sequece teds? What ca you say about the sequece a )? For ǫ = 0.,
More informationSequences and Series
CHAPTER 9 Sequeces ad Series 9.. Covergece: Defiitio ad Examples Sequeces The purpose of this chapter is to itroduce a particular way of geeratig algorithms for fidig the values of fuctios defied by their
More informationORDERS OF GROWTH KEITH CONRAD
ORDERS OF GROWTH KEITH CONRAD Itroductio Gaiig a ituitive feel for the relative growth of fuctios is importat if you really wat to uderstad their behavior It also helps you better grasp topics i calculus
More informationFourier Series and the Wave Equation Part 2
Fourier Series ad the Wave Equatio Part There are two big ideas i our work this week. The first is the use of liearity to break complicated problems ito simple pieces. The secod is the use of the symmetries
More informationLecture 4: Cauchy sequences, BolzanoWeierstrass, and the Squeeze theorem
Lecture 4: Cauchy sequeces, BolzaoWeierstrass, ad the Squeeze theorem The purpose of this lecture is more modest tha the previous oes. It is to state certai coditios uder which we are guarateed that limits
More informationSAMPLE QUESTIONS FOR FINAL EXAM. (1) (2) (3) (4) Find the following using the definition of the Riemann integral: (2x + 1)dx
SAMPLE QUESTIONS FOR FINAL EXAM REAL ANALYSIS I FALL 006 3 4 Fid the followig usig the defiitio of the Riema itegral: a 0 x + dx 3 Cosider the partitio P x 0 3, x 3 +, x 3 +,......, x 3 3 + 3 of the iterval
More informationif A S, then X \ A S, and if (A n ) n is a sequence of sets in S, then n A n S,
Lecture 5: Borel Sets Topologically, the Borel sets i a topological space are the σalgebra geerated by the ope sets. Oe ca build up the Borel sets from the ope sets by iteratig the operatios of complemetatio
More informationx(x 1)(x 2)... (x k + 1) = [x] k n+m 1
1 Coutig mappigs For every real x ad positive iteger k, let [x] k deote the fallig factorial ad x(x 1)(x 2)... (x k + 1) ( ) x = [x] k k k!, ( ) k = 1. 0 I the sequel, X = {x 1,..., x m }, Y = {y 1,...,
More informationInfinite Sequences and Series
CHAPTER 4 Ifiite Sequeces ad Series 4.1. Sequeces A sequece is a ifiite ordered list of umbers, for example the sequece of odd positive itegers: 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29...
More information4 n. n 1. You shold think of the Ratio Test as a generalization of the Geometric Series Test. For example, if a n ar n is a geometric sequence then
SECTION 2.6 THE RATIO TEST 79 2.6. THE RATIO TEST We ow kow how to hadle series which we ca itegrate (the Itegral Test), ad series which are similar to geometric or pseries (the Compariso Test), but of
More informationChapter 6: Variance, the law of large numbers and the MonteCarlo method
Chapter 6: Variace, the law of large umbers ad the MoteCarlo method Expected value, variace, ad Chebyshev iequality. If X is a radom variable recall that the expected value of X, E[X] is the average value
More informationSECTION 1.5 : SUMMATION NOTATION + WORK WITH SEQUENCES
SECTION 1.5 : SUMMATION NOTATION + WORK WITH SEQUENCES Read Sectio 1.5 (pages 5 9) Overview I Sectio 1.5 we lear to work with summatio otatio ad formulas. We will also itroduce a brief overview of sequeces,
More informationChapter 7 Methods of Finding Estimators
Chapter 7 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 011 Chapter 7 Methods of Fidig Estimators Sectio 7.1 Itroductio Defiitio 7.1.1 A poit estimator is ay fuctio W( X) W( X1, X,, X ) of
More informationRepeating Decimals are decimal numbers that have number(s) after the decimal point that repeat in a pattern.
5.5 Fractios ad Decimals Steps for Chagig a Fractio to a Decimal. Simplify the fractio, if possible. 2. Divide the umerator by the deomiator. d d Repeatig Decimals Repeatig Decimals are decimal umbers
More information13 Fast Fourier Transform (FFT)
13 Fast Fourier Trasform FFT) The fast Fourier trasform FFT) is a algorithm for the efficiet implemetatio of the discrete Fourier trasform. We begi our discussio oce more with the cotiuous Fourier trasform.
More informationExample 2 Find the square root of 0. The only square root of 0 is 0 (since 0 is not positive or negative, so those choices don t exist here).
BEGINNING ALGEBRA Roots ad Radicals (revised summer, 00 Olso) Packet to Supplemet the Curret Textbook  Part Review of Square Roots & Irratioals (This portio ca be ay time before Part ad should mostly
More informationSection 11.3: The Integral Test
Sectio.3: The Itegral Test Most of the series we have looked at have either diverged or have coverged ad we have bee able to fid what they coverge to. I geeral however, the problem is much more difficult
More informationA Gentle Introduction to Algorithms: Part II
A Getle Itroductio to Algorithms: Part II Cotets of Part I:. Merge: (to merge two sorted lists ito a sigle sorted list.) 2. Bubble Sort 3. Merge Sort: 4. The BigO, BigΘ, BigΩ otatios: asymptotic bouds
More informationDiscrete Mathematics and Probability Theory Spring 2014 Anant Sahai Note 13
EECS 70 Discrete Mathematics ad Probability Theory Sprig 2014 Aat Sahai Note 13 Itroductio At this poit, we have see eough examples that it is worth just takig stock of our model of probability ad may
More informationThe Euler Totient, the Möbius and the Divisor Functions
The Euler Totiet, the Möbius ad the Divisor Fuctios Rosica Dieva July 29, 2005 Mout Holyoke College South Hadley, MA 01075 1 Ackowledgemets This work was supported by the Mout Holyoke College fellowship
More informationContinued Fractions continued. 3. Best rational approximations
Cotiued Fractios cotiued 3. Best ratioal approximatios We hear so much about π beig approximated by 22/7 because o other ratioal umber with deomiator < 7 is closer to π. Evetually 22/7 is defeated by 333/06
More informationSoving Recurrence Relations
Sovig Recurrece Relatios Part 1. Homogeeous liear 2d degree relatios with costat coefficiets. Cosider the recurrece relatio ( ) T () + at ( 1) + bt ( 2) = 0 This is called a homogeeous liear 2d degree
More informationRecursion and Recurrences
Chapter 5 Recursio ad Recurreces 5.1 Growth Rates of Solutios to Recurreces Divide ad Coquer Algorithms Oe of the most basic ad powerful algorithmic techiques is divide ad coquer. Cosider, for example,
More information7. Sample Covariance and Correlation
1 of 8 7/16/2009 6:06 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 7. Sample Covariace ad Correlatio The Bivariate Model Suppose agai that we have a basic radom experimet, ad that X ad Y
More informationPage 2 of 14 = T(2) + 2 = [ T(3)+1 ] + 2 Substitute T(3)+1 for T(2) = T(3) + 3 = [ T(4)+1 ] + 3 Substitute T(4)+1 for T(3) = T(4) + 4 After i
Page 1 of 14 Search C455 Chapter 4  Recursio Tree Documet last modified: 02/09/2012 18:42:34 Uses: Use recursio tree to determie a good asymptotic boud o the recurrece T() = Sum the costs withi each level
More informationCS103X: Discrete Structures Homework 4 Solutions
CS103X: Discrete Structures Homewor 4 Solutios Due February 22, 2008 Exercise 1 10 poits. Silico Valley questios: a How may possible sixfigure salaries i whole dollar amouts are there that cotai at least
More information0.7 0.6 0.2 0 0 96 96.5 97 97.5 98 98.5 99 99.5 100 100.5 96.5 97 97.5 98 98.5 99 99.5 100 100.5
Sectio 13 KolmogorovSmirov test. Suppose that we have a i.i.d. sample X 1,..., X with some ukow distributio P ad we would like to test the hypothesis that P is equal to a particular distributio P 0, i.e.
More informationMaximum Likelihood Estimators.
Lecture 2 Maximum Likelihood Estimators. Matlab example. As a motivatio, let us look at oe Matlab example. Let us geerate a radom sample of size 00 from beta distributio Beta(5, 2). We will lear the defiitio
More informationModule 4: Mathematical Induction
Module 4: Mathematical Iductio Theme 1: Priciple of Mathematical Iductio Mathematical iductio is used to prove statemets about atural umbers. As studets may remember, we ca write such a statemet as a predicate
More informationReview for College Algebra Final Exam
Review for College Algebra Fial Exam (Please remember that half of the fial exam will cover chapters 14. This review sheet covers oly the ew material, from chapters 5 ad 7.) 5.1 Systems of equatios i
More informationChapter 5: Inner Product Spaces
Chapter 5: Ier Product Spaces Chapter 5: Ier Product Spaces SECION A Itroductio to Ier Product Spaces By the ed of this sectio you will be able to uderstad what is meat by a ier product space give examples
More informationKey Ideas Section 81: Overview hypothesis testing Hypothesis Hypothesis Test Section 82: Basics of Hypothesis Testing Null Hypothesis
Chapter 8 Key Ideas Hypothesis (Null ad Alterative), Hypothesis Test, Test Statistic, Pvalue Type I Error, Type II Error, Sigificace Level, Power Sectio 81: Overview Cofidece Itervals (Chapter 7) are
More informationLecture Notes CMSC 251
We have this messy summatio to solve though First observe that the value remais costat throughout the sum, ad so we ca pull it out frot Also ote that we ca write 3 i / i ad (3/) i T () = log 3 (log ) 1
More informationIncremental calculation of weighted mean and variance
Icremetal calculatio of weighted mea ad variace Toy Fich faf@cam.ac.uk dot@dotat.at Uiversity of Cambridge Computig Service February 009 Abstract I these otes I eplai how to derive formulae for umerically
More informationHypothesis testing. Null and alternative hypotheses
Hypothesis testig Aother importat use of samplig distributios is to test hypotheses about populatio parameters, e.g. mea, proportio, regressio coefficiets, etc. For example, it is possible to stipulate
More informationProject Deliverables. CS 361, Lecture 28. Outline. Project Deliverables. Administrative. Project Comments
Project Deliverables CS 361, Lecture 28 Jared Saia Uiversity of New Mexico Each Group should tur i oe group project cosistig of: About 612 pages of text (ca be loger with appedix) 612 figures (please
More informationFIBONACCI NUMBERS: AN APPLICATION OF LINEAR ALGEBRA. 1. Powers of a matrix
FIBONACCI NUMBERS: AN APPLICATION OF LINEAR ALGEBRA. Powers of a matrix We begi with a propositio which illustrates the usefuless of the diagoalizatio. Recall that a square matrix A is diogaalizable if
More information1 Computing the Standard Deviation of Sample Means
Computig the Stadard Deviatio of Sample Meas Quality cotrol charts are based o sample meas ot o idividual values withi a sample. A sample is a group of items, which are cosidered all together for our aalysis.
More informationChapter Gaussian Elimination
Chapter 04.06 Gaussia Elimiatio After readig this chapter, you should be able to:. solve a set of simultaeous liear equatios usig Naïve Gauss elimiatio,. lear the pitfalls of the Naïve Gauss elimiatio
More information5: Introduction to Estimation
5: Itroductio to Estimatio Cotets Acroyms ad symbols... 1 Statistical iferece... Estimatig µ with cofidece... 3 Samplig distributio of the mea... 3 Cofidece Iterval for μ whe σ is kow before had... 4 Sample
More informationU.C. Berkeley CS270: Algorithms Lecture 9 Professor Vazirani and Professor Rao Last revised. Lecture 9
U.C. Berkeley CS270: Algorithms Lecture 9 Professor Vazirai a Professor Rao Scribe: Aupam Last revise Lecture 9 1 Sparse cuts a Cheeger s iequality Cosier the problem of partitioig a give graph G(V, E
More informationHere are a couple of warnings to my students who may be here to get a copy of what happened on a day that you missed.
This documet was writte ad copyrighted by Paul Dawkis. Use of this documet ad its olie versio is govered by the Terms ad Coditios of Use located at http://tutorial.math.lamar.edu/terms.asp. The olie versio
More information1 n. n > dt. t < n 1 + n=1
Math 05 otes C. Pomerace The harmoic sum The harmoic sum is the sum of recirocals of the ositive itegers. We kow from calculus that it diverges, this is usually doe by the itegral test. There s a more
More informationDepartment of Computer Science, University of Otago
Departmet of Computer Sciece, Uiversity of Otago Techical Report OUCS200609 Permutatios Cotaiig May Patters Authors: M.H. Albert Departmet of Computer Sciece, Uiversity of Otago Micah Colema, Rya Fly
More information1 The Binomial Theorem: Another Approach
The Biomial Theorem: Aother Approach Pascal s Triagle I class (ad i our text we saw that, for iteger, the biomial theorem ca be stated (a + b = c a + c a b + c a b + + c ab + c b, where the coefficiets
More informationConfidence Intervals for the Mean of Nonnormal Data Class 23, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom
Cofidece Itervals for the Mea of Noormal Data Class 23, 8.05, Sprig 204 Jeremy Orloff ad Joatha Bloom Learig Goals. Be able to derive the formula for coservative ormal cofidece itervals for the proportio
More informationSection 9.2 Series and Convergence
Sectio 9. Series ad Covergece Goals of Chapter 9 Approximate Pi Prove ifiite series are aother importat applicatio of limits, derivatives, approximatio, slope, ad cocavity of fuctios. Fid challegig atiderivatives
More informationSolving DivideandConquer Recurrences
Solvig DivideadCoquer Recurreces Victor Adamchik A divideadcoquer algorithm cosists of three steps: dividig a problem ito smaller subproblems solvig (recursively) each subproblem the combiig solutios
More informationTHE LEAST SQUARES REGRESSION LINE and R 2
THE LEAST SQUARES REGRESSION LINE ad R M358K I. Recall from p. 36 that the least squares regressio lie of y o x is the lie that makes the sum of the squares of the vertical distaces of the data poits from
More informationA probabilistic proof of a binomial identity
A probabilistic proof of a biomial idetity Joatho Peterso Abstract We give a elemetary probabilistic proof of a biomial idetity. The proof is obtaied by computig the probability of a certai evet i two
More informationApproximating the Sum of a Convergent Series
Approximatig the Sum of a Coverget Series Larry Riddle Ages Scott College Decatur, GA 30030 lriddle@agesscott.edu The BC Calculus Course Descriptio metios how techology ca be used to explore covergece
More informationLesson 15 ANOVA (analysis of variance)
Outlie Variability betwee group variability withi group variability total variability Fratio Computatio sums of squares (betwee/withi/total degrees of freedom (betwee/withi/total mea square (betwee/withi
More information8.5 Alternating infinite series
65 8.5 Alteratig ifiite series I the previous two sectios we cosidered oly series with positive terms. I this sectio we cosider series with both positive ad egative terms which alterate: positive, egative,
More informationFactoring x n 1: cyclotomic and Aurifeuillian polynomials Paul Garrett <garrett@math.umn.edu>
(March 16, 004) Factorig x 1: cyclotomic ad Aurifeuillia polyomials Paul Garrett Polyomials of the form x 1, x 3 1, x 4 1 have at least oe systematic factorizatio x 1 = (x 1)(x 1
More information1 Correlation and Regression Analysis
1 Correlatio ad Regressio Aalysis I this sectio we will be ivestigatig the relatioship betwee two cotiuous variable, such as height ad weight, the cocetratio of a ijected drug ad heart rate, or the cosumptio
More informationThe Stable Marriage Problem
The Stable Marriage Problem William Hut Lae Departmet of Computer Sciece ad Electrical Egieerig, West Virgiia Uiversity, Morgatow, WV William.Hut@mail.wvu.edu 1 Itroductio Imagie you are a matchmaker,
More informationThe Field Q of Rational Numbers
Chapter 3 The Field Q of Ratioal Numbers I this chapter we are goig to costruct the ratioal umber from the itegers. Historically, the positive ratioal umbers came first: the Babyloias, Egyptias ad Grees
More information7 b) 0. Guided Notes for lesson P.2 Properties of Exponents. If a, b, x, y and a, b, 0, and m, n Z then the following properties hold: 1 n b
Guided Notes for lesso P. Properties of Expoets If a, b, x, y ad a, b, 0, ad m, Z the the followig properties hold:. Negative Expoet Rule: b ad b b b Aswers must ever cotai egative expoets. Examples: 5
More informationCS103A Handout 23 Winter 2002 February 22, 2002 Solving Recurrence Relations
CS3A Hadout 3 Witer 00 February, 00 Solvig Recurrece Relatios Itroductio A wide variety of recurrece problems occur i models. Some of these recurrece relatios ca be solved usig iteratio or some other ad
More informationThe geometric series and the ratio test
The geometric series ad the ratio test Today we are goig to develop aother test for covergece based o the iterplay betwee the it compariso test we developed last time ad the geometric series. A ote about
More informationarxiv:1012.1336v2 [cs.cc] 8 Dec 2010
Uary SubsetSum is i Logspace arxiv:1012.1336v2 [cs.cc] 8 Dec 2010 1 Itroductio Daiel M. Kae December 9, 2010 I this paper we cosider the Uary SubsetSum problem which is defied as follows: Give itegers
More informationTIEE Teaching Issues and Experiments in Ecology  Volume 1, January 2004
TIEE Teachig Issues ad Experimets i Ecology  Volume 1, Jauary 2004 EXPERIMENTS Evirometal Correlates of Leaf Stomata Desity Bruce W. Grat ad Itzick Vatick Biology, Wideer Uiversity, Chester PA, 19013
More informationLesson 12. Sequences and Series
Retur to List of Lessos Lesso. Sequeces ad Series A ifiite sequece { a, a, a,... a,...} ca be thought of as a list of umbers writte i defiite order ad certai patter. It is usually deoted by { a } =, or
More informationThe following example will help us understand The Sampling Distribution of the Mean. C1 C2 C3 C4 C5 50 miles 84 miles 38 miles 120 miles 48 miles
The followig eample will help us uderstad The Samplig Distributio of the Mea Review: The populatio is the etire collectio of all idividuals or objects of iterest The sample is the portio of the populatio
More informationTrigonometric Form of a Complex Number. The Complex Plane. axis. ( 2, 1) or 2 i FIGURE 6.44. The absolute value of the complex number z a bi is
0_0605.qxd /5/05 0:45 AM Page 470 470 Chapter 6 Additioal Topics i Trigoometry 6.5 Trigoometric Form of a Complex Number What you should lear Plot complex umbers i the complex plae ad fid absolute values
More informationDefinition. Definition. 72 Estimating a Population Proportion. Definition. Definition
7 stimatig a Populatio Proportio I this sectio we preset methods for usig a sample proportio to estimate the value of a populatio proportio. The sample proportio is the best poit estimate of the populatio
More informationLecture 5: Span, linear independence, bases, and dimension
Lecture 5: Spa, liear idepedece, bases, ad dimesio Travis Schedler Thurs, Sep 23, 2010 (versio: 9/21 9:55 PM) 1 Motivatio Motivatio To uderstad what it meas that R has dimesio oe, R 2 dimesio 2, etc.;
More informationLECTURE 13: Crossvalidation
LECTURE 3: Crossvalidatio Resampli methods Cross Validatio Bootstrap Bias ad variace estimatio with the Bootstrap Threeway data partitioi Itroductio to Patter Aalysis Ricardo GutierrezOsua Texas A&M
More informationf(x + T ) = f(x), for all x. The period of the function f(t) is the interval between two successive repetitions.
Fourier Series. Itroductio Whe the Frech mathematicia Joseph Fourier (76883) was tryig to study the flow of heat i a metal plate, he had the idea of expressig the heat source as a ifiite series of sie
More informationDivide and Conquer. Maximum/minimum. Integer Multiplication. CS125 Lecture 4 Fall 2015
CS125 Lecture 4 Fall 2015 Divide ad Coquer We have see oe geeral paradigm for fidig algorithms: the greedy approach. We ow cosider aother geeral paradigm, kow as divide ad coquer. We have already see a
More informationOur aim is to show that under reasonable assumptions a given 2πperiodic function f can be represented as convergent series
8 Fourier Series Our aim is to show that uder reasoable assumptios a give periodic fuctio f ca be represeted as coverget series f(x) = a + (a cos x + b si x). (8.) By defiitio, the covergece of the series
More information1. C. The formula for the confidence interval for a population mean is: x t, which was
s 1. C. The formula for the cofidece iterval for a populatio mea is: x t, which was based o the sample Mea. So, x is guarateed to be i the iterval you form.. D. Use the rule : pvalue
More information1 The Gaussian channel
ECE 77 Lecture 0 The Gaussia chael Objective: I this lecture we will lear about commuicatio over a chael of practical iterest, i which the trasmitted sigal is subjected to additive white Gaussia oise.
More informationGibbs Distribution in Quantum Statistics
Gibbs Distributio i Quatum Statistics Quatum Mechaics is much more complicated tha the Classical oe. To fully characterize a state of oe particle i Classical Mechaics we just eed to specify its radius
More informationUniversity of California, Los Angeles Department of Statistics. Distributions related to the normal distribution
Uiversity of Califoria, Los Ageles Departmet of Statistics Statistics 100B Istructor: Nicolas Christou Three importat distributios: Distributios related to the ormal distributio Chisquare (χ ) distributio.
More information3.1 Measures of Central Tendency. Introduction 5/28/2013. Data Description. Outline. Objectives. Objectives. Traditional Statistics Average
5/8/013 C H 3A P T E R Outlie 3 1 Measures of Cetral Tedecy 3 Measures of Variatio 3 3 3 Measuresof Positio 3 4 Exploratory Data Aalysis Copyright 013 The McGraw Hill Compaies, Ic. C H 3A P T E R Objectives
More informationStatistics Lecture 14. Introduction to Inference. Administrative Notes. Hypothesis Tests. Last Class: Confidence Intervals
Statistics 111  Lecture 14 Itroductio to Iferece Hypothesis Tests Admiistrative Notes Sprig Break! No lectures o Tuesday, March 8 th ad Thursday March 10 th Exteded Sprig Break! There is o Stat 111 recitatio
More informationSearching Algorithm Efficiencies
Efficiecy of Liear Search Searchig Algorithm Efficiecies Havig implemeted the liear search algorithm, how would you measure its efficiecy? A useful measure (or metric) should be geeral, applicable to ay
More informationLearning outcomes. Algorithms and Data Structures. Time Complexity Analysis. Time Complexity Analysis How fast is the algorithm? Prof. Dr.
Algorithms ad Data Structures Algorithm efficiecy Learig outcomes Able to carry out simple asymptotic aalysisof algorithms Prof. Dr. Qi Xi 2 Time Complexity Aalysis How fast is the algorithm? Code the
More informationSection 73 Estimating a Population. Requirements
Sectio 73 Estimatig a Populatio Mea: σ Kow Key Cocept This sectio presets methods for usig sample data to fid a poit estimate ad cofidece iterval estimate of a populatio mea. A key requiremet i this sectio
More information8.3 POLAR FORM AND DEMOIVRE S THEOREM
SECTION 8. POLAR FORM AND DEMOIVRE S THEOREM 48 8. POLAR FORM AND DEMOIVRE S THEOREM Figure 8.6 (a, b) b r a 0 θ Complex Number: a + bi Rectagular Form: (a, b) Polar Form: (r, θ) At this poit you ca add,
More information{{1}, {2, 4}, {3}} {{1, 3, 4}, {2}} {{1}, {2}, {3, 4}} 5.4 Stirling Numbers
. Stirlig Numbers Whe coutig various types of fuctios from., we quicly discovered that eumeratig the umber of oto fuctios was a difficult problem. For a domai of five elemets ad a rage of four elemets,
More informationDetermining the sample size
Determiig the sample size Oe of the most commo questios ay statisticia gets asked is How large a sample size do I eed? Researchers are ofte surprised to fid out that the aswer depeds o a umber of factors
More information1.3 Binomial Coefficients
18 CHAPTER 1. COUNTING 1. Biomial Coefficiets I this sectio, we will explore various properties of biomial coefficiets. Pascal s Triagle Table 1 cotais the values of the biomial coefficiets ( ) for 0to
More information