U.C. Berkeley CS270: Algorithms Lecture 9 Professor Vazirani and Professor Rao Last revised. Lecture 9

Transcription

1 U.C. Berkeley CS270: Algorithms Lecture 9 Professor Vazirai a Professor Rao Scribe: Aupam Last revise Lecture 9 1 Sparse cuts a Cheeger s iequality Cosier the problem of partitioig a give graph G(V, E ito two or more large pieces by eletig a few eges. If eges i the graph represet some otio of similarity or closeess betwee the vertices, the such a ecompositio yiels a goo clusterig of the vertices i the graph. The quality of such a ecompositio ca be quatifie i terms of the ege expasio which is efie ext. For simplicity we assume that G is a -regular graph, let (S, V S be a partitio of the vertex set V. The ege expasio of the partitio (S, V S is, h(s = E(S, V S mi( S, V S (1 The ege expasio of the graph G is the miimum expasio over all partitios of V, h(g = mi h(s (2 S V Aother way to escribe the ege expasio is the followig: We wish to break off a large part of the graph G by cuttig oly a small umber of eges. Give that we get to keep the smaller of the two pieces, the ege expasio is the ratio of the umber of eges cut to the umber of eges that we get to keep. (icluig the cut eges. The couctace φ(s of a partitio (S, V S is a measure closely relate to the ege expasio h(s. The couctace of G is the miimum couctace over all partitios of V, φ(g = mi S V E(S, V S S V S The couctace of a partitio (S, V S is the ege expasio h(s multiplie by / S where S /2 is the size of the larger piece. The couctace approximates the ege expasio withi a factor of 2 for all partitios, so reasoig about the two measures is equivalet. (3 h(s φ(s 2h(S (4 The ege expasio of a graph is closely relate to the seco largest eigevalue of its ajacecy matrix. The coectio was first iscovere by Cheeger i ifferetial geometry, relatig the isoperimetric properties of Riemaia maifols to the seco eigevalue of Laplacia operators efie o them. The results were itrouce to computer sciece by Alo, a have fou several applicatios sice the icluig work by Jerrum a Siclair o bouig the mixig time of Markov chais a by Malik a Shi o image segmetatio.

2 Notes for Lecture 9: Scribe: Aupam The spectrum of a graph The ajacecy matrix of G is the matrix with etry A ij = 1 if (i, j is a ege a 0 otherwise. The ajacecy matrix is ormalize to M = A/ such that all the rows of M sum to 1. The matrix M is real a symmetric so it has a orthoormal set of eigevectors v i with correspoig eigevalues λ i for i [] by the spectral theorem. Wlog we ca assume that the eigevectors are real i.e. v i R. Claim 1 If v, v are eigevectors of a real symmetric matrix M with istict eigevalues λ, λ the v t v = 0. Proof: The proof follows by evaluatig the expressio v t Mv i two ifferet ways usig the fact that M T = M, v t Mv = λ v t v v t Mv = M T v t v = λv t v As λ, λ are istict, the vectors v, v must be orthogoal. The claim says that there is a basis of eigevectors v i i which the actio of M is to shrik or expa the basis vectors. The matrix M is iagoal i the basis of eigevectors, λ λ M =..... ( λ The basis of eigevectors is uiquely etermie if all the eigevalues are istict. If a eigevalue λ i has multiplicity the space of vectors with eigevalue λ i has imesio. A basis of eigevectors ca be efie by choosig ay orthoormal basis for each of the eigespaces. 1.2 Actio of the ajacecy matrix It is useful to visualize the actio of M as follows: A vector v R ca be though of as assigig a weight v i to vertex i i G. The actio of M maps v to Mv, which replaces v i by j i v j. Therefore M acts by replacig the weight of each vertex by the average weight 1 of its eighbors. We have itrouce three ifferet cocepts so far: sparse cuts i G, eigevectors a eigevalues of the ormalize ajacecy matrix M a the actio of the ormalize ajacecy matrix. We will relate the three views, rawig o the ifferet perspectives to prove results. The vector v 1 with all etries equal to 1 is a eigevector for a regular graph with eigevalue 1. The seco largest eigevalue λ 2 is strictly less tha 1 if G is coecte,

3 Notes for Lecture 9: Scribe: Aupam 3 Claim 2 λ 1 = 1, v 1 = 1 1 a λ k 1 for a regular graph G. If the graph G is coecte, λ 2 < 1. Proof: The vector v 1 = 1 1 has uit legth a Mv = v by the averagig iterpretatio of the the actio of M, showig that λ 1 = 1. The cooriates of the k-th eigevector v k scale by a factor of λ k uer averagig over eighbors, (λ k v k i = (Mv k i = 1 v kj (6 Let i be a cooriate of v k havig the maximum absolute value, by the triagle iequality we have, λ k v ki 1 v kj v ki (7 j i It follows that λ k 1 for all k []. Suppose v is a eigevector with eigevalue 1 for a coecte graph G. Equality hols i (7 for v, so v j = v i for all vertices j ajacet to i. All vertices ca be reache by paths startig at i as G is a coecte graph so v k = v i for all k. It follows that a coecte graph has a uique eigevector with eigevalue 1, hece λ 2 < 1. The above claim was prove usig the averagig iterpretatio of the actio of M. The iagoal represetatio of M i the spectral basis (5 yiels aother characterizatio of the eigevalues, Claim 3 x T Mx λ 1 = sup x R x t x Proof: The matrix M is iagoal i the spectral basis (5 so x T Mx ca be evaluate easily if x is represete i the spectral basis, x t Mx = λ i x 2 i λ 1 x 2 i = λ 1 x t x (8 i i The iequality is tight for x 1 = e 1 i.e. whe x is the first eigevector. The quatity xt Mx is calle the Rayleigh quotiet i the literature. A argumet similar x T x to (8 for vectors x such that x 1 = 0 i the spectral basis provies a characterizatio of the seco eigevalue, x s.t. x 1 = 0, x t Mx = λ i x 2 i λ 2 x 2 i = λ 2 x t x (9 i>1 i The iequality is tight for x = e 2 i.e. whe x is the seco eigevector. j i

4 Notes for Lecture 9: Scribe: Aupam 4 The first eigevector for a regular graph is 1 1 so x 1 = 0 i the spectral basis is equivalet to sayig that x 1 i the staar basis. Substitutig i (9 we have the Rayleigh quotiet characterizatio of the seco eigevalue, x T Mx λ 2 = sup x 1 x t x More geerally, all the eigevalues have a similar characterizatio give by the Courat Fischer theorem (the proof relies o a imesio argumet i liear algebra, exercise (10 λ k = max mi x T Mx im(s=k x S x t x ( Cheeger s iequality The first eigevector of a regular graph is 1 1 a oes ot reveal iformatio about the graph structure. The spectral gap µ = λ 1 λ 2 is the ifferece betwee the first two eigevalues. The spectral gap reveals iformatio about the coectivity, for example µ = 0 if a oly if G has more tha oe coecte compoet. Cheeger s iequality provies the coectio betwee the spectral gap a ege expasio, µ 2 = 1 λ 2 2 h(g 2(1 λ 2 = 2µ (12 We illustrate Cheeger s iequality with the examples of the imesioal hypercube a the cycle, showig that both sies of the iequality are tight. The iequality will be prove over the ext few lectures. 1.4 The hypercube The imesioal hypercube has V = {0, 1} with (x, y E if x a y represete as biary strigs iffer i exactly oe bit. The umber of vertices is = 2, each vertex has egree so the umber of eges is 2 1. A way to picture the hypercube is the followig: The imesioal hypercube is built by takig two copies of a 1 imesioal hypercube a coectig the correspoig vertices. Ege expasio: The i-th cooriate cut i the hypercube is efie as S i := {x {0, 1} x i = 0}. The cooriate cuts achieve the miimum value for the ege expasio. This will follow from Cheeger s iequality oce we obtai the spectrum of the hypercube, h(g = E(S i, S i S i = = 1 = 1 log (13 Vertex expasio: The ball cut S := {x {0, 1} x i /2} cosists of strigs that have at most /2 oes. The cut S achieves the smallest possible vertex expasio: more geerally a isoperimetric theorem for the hypercube says that a cut havig 1+ ( ( k vertices must have at least ( k vertices at its bouary. If the vertices of the hypercube are permute raomly it turs out that is is ifficult to recostruct the cooriate cuts, however it is easy to recover goo approximatios to the ball cuts. What is the ege expasio achieve by the ball cuts?

5 Notes for Lecture 9: Scribe: Aupam 5 A vertex at the bouary of S has exactly /2 eighbors i S, so the ege expasio is give by, E(S, S 1 h(s = (14 S i log = S 2 1 = The size of S is equal to ( /2 2 where the approximatio log! log is use. [The approximatio is kow as Stirlig s formula i the literature, exercise if you have ot see this before] The spectrum of the hypercube: Fiig the spectrum: Recall that there are ( 1 cooriate cuts o the hypercube give by S i = {x {0, 1} x i = 0}. Cosier the characteristic vectors v i of the cooriate cuts where v i (x = 1 if x S i a v i (x = 1 if x S i. Each vertex i S i has 1 eighbors i S i a oe eighbor i S i, the averagig iterpretatio of the actio of the ajacecy matrix shows that Mv i = (1 2/v i. The cooriate cuts v i are eigevectors of the hypercube with eigevalue 1 2/. Cosier the ( 2 cuts o the hypercube give by Sij = {x {0, 1} (x i x j = 1}. These cuts are obtaie by cosierig the 4 hypercubes of imesio 2 cotaie i the imesioal hypercube. Each vertex i S ij has 2 eighbors i S ij a two eighbors i S ij, the averagig iterpretatio of the actio of the ajacecy matrix shows that Mv ij = (1 4/v ij, where v ij is the characteristic vector of S ij. The vectors v ij are mutually orthogoal, showig that there are ( 2 eigevectors for the hypercube with eigevalue 1 4/. Similarly by lookig at the imesio k hypercubes isie the imesioal hypercube we fi ( k eigevectors with eigevalue 1 2k/. The sum of the biomial coefficiets ( i = 2, so we have fou the complete spectrum of the hypercube. A histogram of the spectral profile of the hypercube looks like a plot of the biomial istributio scale to lie i [ 1, 1]. A cleaer way to obtai the spectrum of the hypercube is to observe that the hypercube is a Cayley graph for the group Z 2, the spectra of Cayley graphs ca be etermie easily as the eigevectors are the Fourier basis vectors (exercise for theorists. The seco eigespace: The seco eigespace of the hypercube has imesio a is spae by the cooriate cuts. If the hypercube is raomly permute so that the vertex labels are lost, a eigevalue fiig program will output some liear combiatio of the cooriate cuts as the seco eigevector. The ball cut i (14 is a liear combiatio of the cooriate cuts with coefficiets 1/2. A raom liear combiatio of the cooriate cuts has expasio similar to the ball cuts. Tightess of Cheeger s iequality: The hypercube is a example for which the left sie of Cheeger s iequality is tight. We showe that 1 λ 2 2 = 1 a equatio (13 shows that the ege expasio of the cooriate cuts is h(s i = 1. Equality hols i the left sie of Cheeger s iequality for the hypercube. It follows that the cooriate cuts are the sparsest cuts for the hypercube.

6 Notes for Lecture 9: Scribe: Aupam The cycle The cycle has vertex set [] a eges (i, i + 1 mo for i []. The ege expasio of the cycle h(c = 2/ a the sparsest cut is the partitio of the cycle ito two equal halves. We will show that λ 2 1 O( 1 for the cycle, implyig that the right sie of 2 Cheeger s iequality h(c 2(1 λ 2 is tight for the cycle. Proof of the tightess of Cheeger: A quick way to show that the seco eigevector for the cycle is large is to fi a vector x 1 havig a high Rayleigh quotiet. A vector with high Rayleigh quotiet will be approximately ivariat uer averagig over eighbors. Cosier the followig vector, { i /4 if i /2 x i = 3/4 i if i > /2 The vector x is chose to be orthogoal to 1 a the cooriates of x are approximately ivariat uer averagig over eighbors, /4 + 1/2 if i = 1, (Mx i = /4 1 if i = /2 otherwise x i Usig x t x = i [/4] 16i2 = O( 3 the value of the Rayleigh quotiet ca be compute as follows, x t ( Mx x t x = xt x O( 1 x t = 1 O x 2 The seco eigevalue λ 2 must be greater tha the Rayleigh quotiet of x, showig that the right sie of Cheeger s iequality is tight for the cycle. Alterative proof by computig the spectrum: We will write ow the eigevalues a eigevectors for the cycle explicitly. A cleaer way to obtai the spectrum of the cycle is to observe that it is a Cayley graph for Z a the eigevectors are Fourier basis vectors. The eigevalues of the cycle are cos(2πk/ for 0 k 1. The vector v with cooriates v i = cos(2πki/ is a eigevector of the cycle with eigevalue cos(2πk/. This follows from the trigoometric ietity cos(a + B = cos A cos B si A si B, ( ( ( ( 2πk(i + 1 2πk(i 1 2πk 2πki cos + cos = 2 cos cos The vector w with cooriates w i = si(2πki/ is also a eigevector of the cycle with eigevalue cos(2πk/, this ca be see usig the ietity si(a + B = si A cos B + cos A si B. The eigespaces of the cycle correspoig to o ±1 eigevalues are two imesioal, this also follows from the fact that the eigevectors of the cycle are the Fourier basis vectors. More geerally graphs with symmetries have egeerate eigespaces, the staar embeig of the cycle has a reflectio symmetry about the x axis. Tightess of Cheeger s iequality: As, the spectral gap for the cycle tes to 1 2 cos(2π/ = O(1/ 2 usig the Taylor series expasio cos(δ = 1 δ2 2 + o(δ2 for δ 0.