# 3. Markov property. Markov property for MRFs. Hammersley-Clifford theorem. Markov property for Bayesian networks. I-map, P-map, and chordal graphs

Save this PDF as:

Size: px
Start display at page:

Download "3. Markov property. Markov property for MRFs. Hammersley-Clifford theorem. Markov property for Bayesian networks. I-map, P-map, and chordal graphs"

## Transcription

1 Markov property 3-3. Markov property Markov property for MRFs Hammersley-Clifford theorem Markov property for ayesian networks I-map, P-map, and chordal graphs

2 Markov Chain X Y Z X = Z Y µ(x, Y, Z) = f(x, Y )g(y, Z) Q. What independence does MRF imply? x 0 x x 3 x x x 4 x x 9 x 5 x 7 x8 x 6 Markov property 3-

3 Markov property 3-3 Markov property A let A C be a partition of V Definition: graph separation C separates A from C if any path starting in A and terminating in C has at least one node in Definition: global Markov property distribution µ over X V satisfies the global Markov property on G if for any partition (A,, C) such that separates A from C, µ(x A, x C x ) = µ(x A x ) µ(x C x )

4 Markov property 3-4 Markov property for undirected graphs We say µ( ) satisfy the global Markov property (G) w.r.t. a graph G if for any partition (A,, C) such that separates A from C, µ(x A, x C x ) = µ(x A x ) µ(x C x ) We say µ( ) satisfy the local Markov property (L) w.r.t. a graph G if for any i V, µ(x i, x rest x i ) = µ(x i x i ) µ(x rest x i ) We say µ( ) satisfy the pairwise Markov property (P) w.r.t. a graph G if for any i, j V that are not connected by an edge obviously: (G) (L) (P) µ(x i, x j x rest ) = µ(x i x rest ) µ(x j x rest )

5 Proof of (L) (P): suppose (L) holds, then for any i and j not connected by an edge. since, for any deterministic function h( ), proofs of () and (): X Y Z = X Y (Z, h(y )) () X Y Z = X h(y ) Z () x i x rest x i () = x i x rest (x V \{i,j} ) () = x i x j (x V \{i,j} ) µ(x, y, h(y ) = h, z) = µ(x, y, z)i(h(y) = h) = f(x, z)g(y, z)i(h(y) = h) this implies both X (Y, h(y )) Z and X Y (Z, h(y )) µ(x, h, z) = y µ(x, y, h, z) = f(x, z) g(y, z)i(h(y) = h) y }{{} g(h,z) this implies X h(y ) Z Markov property 3-5 = = =

6 what conditions do we need for (P ) (G) to hold? Markov property 3-6

7 Markov property 3-7 in general (P ) (G), but (P ) (G) if the following holds for all disjoint subsets A,,C, and D V : intersection lemma if x A x (x C, x D ) and x A x C (x, x D ), then x A (x, x C ) x D (3) for instance, for a strictly positive µ( ), i.e. if µ(x) > 0 for all x X V, then the above holds [Homework.5]

8 Markov property 3-8 Proof of (P) (G) when (3) holds: [Pearl,Paz 987] by induction over s when s = n, (P) (G) assume (G) for any with s and prove it for = s i by induction assumption by (3), x C xã (x, x i ) x C x i (x, xã) Ã A C x C (xã, x i ) x by induction, we see that (G) holds for all sizes of.

9 Markov property 3-9 Hammersley-Clifford Theorem (Hammersley-Clifford) 97 A strictly positive distribution µ(x) (i.e. µ(x) > 0 for all x) satisfies the global Markov property (G) with respect to G(V, E) if and only if it can be factorized according to G: (F) : µ(x) = Z c C(G) ψ c (x c ) i.e. any µ(x) with Markov property can be represented with MRF (F) (G) is easy [Homework.] (F) (G) requires effort

10 Markov property 3-0 Proof sketch (Grimmett, ull. London Math. Soc. 973) for every S V, define ψ S (x S ) U S µ(x U, 0 V \U ) ( ) S\U need strict positive µ( ) for the division to make sense example: For S = {i} let µ + i ( ) µ(, 0 V \{i}) ψ i (x i ) = µ+ i (x i) µ + i (0) for S = {i, j} (not necessarily an edge) let µ + ij ( ) µ(, 0 V \{i,j}) ψ ij (x i, x j ) = µ+ ij (x i, x j ) µ + ij (0, 0) µ + ij (x i, 0) µ + ij (0, x j)

11 Markov property 3- claim : ψs (x S ) =const. unless S is a clique claim : for any µ(x), µ(x) = µ(0 V ) S V ψ S (x S ) then, it follows from claims and that µ(x) = µ(0 V ) c C(G) ψ c (x c ) this proves that for any positive µ(x) satisfying (G) we can find a factorization as per (F )

12 Markov property 3- Proof of claim : ψ S (x S ) = U S µ + U (x U) ( ) S\U i j T S = U T ( µ + U {i,j} (x U, x i, x j ) µ + U (x ) ( ) T \U U) µ + U {i} (x U, x i ) µ + U {j} (x U, x j ) and unless S is a clique, there exists two nodes i and j in S that are not connected by an edge, then µ + U {i,j} (x U, x i, x j ) µ + U {i} (x U, x i ) (G) = µ(x i x U, 0 rest ) µ(x U, x j, 0 rest ) µ(x i x U, 0 rest ) µ(x U, 0 j, 0 rest ) = µ(0 i x U, 0 rest ) µ(x U, x j, 0 rest ) µ(0 i x U, 0 rest ) µ(x U, 0 j, 0 rest ) = µ(x U, 0 i, x j, 0 rest ) µ(x U, 0 i, 0 j, 0 rest ) = µ + U {j} (x U, x j ) µ + U (x U)

13 proof of claim (where did ( ) S\U come from?): recall, ψ S (x S ) U S µ(x U, 0 V \U ) ( ) S\U Möbius inversion lemma [see also G.C.Rota, Prob. Theor. Rel. Fields, (964) ] let f, g : { subsets of V } R. Then the following are equivalent f(s) = U S g(u), for all S V g(s) = U S( ) S\U f(u), for all S V let, f(s) := log µ(x S, 0 V \S ), then g(s) = U S( ) S\U log µ(x U, 0 V \U ) = log ψ S (x S ) ( hence, exp(f(v )) = µ(x) = exp log ψ ) U (x U ) Markov property U V 3-3

14 Give an example of a distribution which is not strictly positive, that does not satisfy (G) [Homework.] Markov property 3-4

15 Markov property 3-5 Recap consider a distribution µ(x) that factorizes according to an undirected graphical model on G = (V, E), µ(x) = ψ c (x c ) Z where C is the set of all maximal cliques in G (Global Markov property) for any disjoint subsets A,, C V, µ(x) satisfy x A x x C whenever separates A and C c C A C

16 Markov property 3-6 x x 3 x 0 x x x 4 x x 9 x 5 x 7 x8 example: color the nodes with {R, G, } such that no adjacent node has the same color µ(x) = I(x i x j ) Z (i,j) E for a node i, if we condition on the color of the neighbors of i, color of i is independent of the rest of the graph x 6

17 Hammersley-Clifford theorem (pairwise) if positive µ(x) satisfies all conditional independences implied by a graph G without any triangles, then we can find a factorization µ(x) = ψ ij (x i, x j ) Z (i,j) E (general) if positive µ(x) satisfies all conditional independences implied by a graph G, then we can find a factorization µ(x) = ψ c (x c ) Z there are conditional independencies that cannot be represented by an undirected graphical model, but is represented by a ayesian network Example: µ(x) = µ(x )µ(x 3 )µ(x x, x 3 ) c C 3 3 x and x 3 are independent no independence Markov property 3-7

18 Markov property 3-8 Markov property of directed graphical models Examples 3 µ(x) = µ(x )µ(x x )µ(x 3 x ) 3 µ(x) = µ(x )µ(x x )µ(x 3 x ) 3 µ(x) = µ(x )µ(x 3 )µ(x x, x 3 ) 3 µ(x) = µ(x )µ(x 3 )µ(x x 3 ) since µ(x 3 x ) = µ(x 3 x, x ), we have x x x 3 since µ(x 3 x ) = µ(x 3 x, x ), we have x x x 3 x and x 3 are independent but not x x x 3 x is independent of (x, x 3 )

19 Markov property 3-9 there are simple independencies we can immediately read from the factorization : x i x pr(i)\π(i) x π(i) (pr(i) is the set of predecessors in a topological ordering) = immediate x x 3 x x 3 x, x 4 x x 4 x, x 3, x 5, x 6 x x 5 x, x 4 x, x 3 x 6 x, x, x 3, x 4 x 5 = = = = = 6 less immediate x 3 x 4 x, but x 3 x 4 x, x 6, etc. = however, there are more independencies that follows from the factorization using the idea of d-separation, there is a way (e.g. ayes ball algorithm) to get all the independencies that are implied by all joint distributions that factorize according to a DAG G

20 note that a particular distribution µ( ) that factorize as G might have more independence than specified by the graph (for example, complete independent distribution (µ(x) = i µ(x i)) always can be represented by any DAG), but we are interested in the family of all µ s that factorize as per G, and statements about the independencies satisfied by all of the µ s in that family Markov property 3-0

21 Markov property 3- formally, let nd(i) be the set of non-descendants of node i, which are the set of nodes that are not reachable from node i via directed paths we say µ(x) has the local Markov property (DL) w.r.t. a directed acyclic graph G, if for all i µ(x i, x nd(i)\π(i) x π(i) ) = µ(x i x π(i) )µ(x nd(i)\π(i) x π(i) ) Definition: topological ordering is an ordering where any node comes after all of its parents (not necessarily unique) let pr(i) be the set of predecessors in a topological ordering we say µ(x) has the ordered Markov property (DO) w.r.t a directed acyclic graph G, if for all i and all topological ordering µ(x i, x pr(i)\π(i) x π(i) ) = µ(x i x π(i) )µ(x pr(i)\π(i) x π(i) ) (DL) (DO), since for each i there exists a topological ordering that places all x nd(i) before x i

22 we say a distribution µ(x) has the global Markov property (DG) w.r.t. an acyclic directed graph G, if for all subsets A,, C such that A and C are d-separated by, two subset A and C are d-separated by if all trails from A to C are by one of the above cases Markov property 3- µ(x A, x C x ) = µ(x A x )µ(x C x ) or equivalently x A = x C x a trail from i to j in a directed acyclic graph is by S if it contains a (shaded) vertex k S such that one of the following happens not not not

23 Markov property x 3 = x 4 x x 3 x 4 x, x 6 A trail from a node x 4 to another node x 6 in a directed graph is any path from x 4 to x 6 that ignores the direction of each of the edges in the path (you can go in the opposite direction of the edge)

24 Markov property 3-4 a ayesian network on a directed acyclic graph G factorizes as (DF) : µ(x) = i V µ i (x i x π(i) ) (DF) (DG) (DL) (DO) proof of (DG) (DL): the non-descendants of a node are d-separated from the node i conditioned on π(i) (proof by example) 3 by (DG) it follows that x 4 = x, x x proof of (DL) (DF): follows from the definition of conditional independence and ayes rule

25 Markov property 3-5 proof of (DF) (DG): requires an alternate (actually the original) definition of global Markov property Lemma : If µ(x) factorizes according to a DAG G, then for any ancestral set A, the marginal distribution µ(x A ) factorizes according to G A, where G A is the subgraph defined from G by deleting all nodes in A C Proof: consider the factorization and marginalize out all not in the ancestral set. Lemma : If µ(x) factorizes according to a DAG G, then x A x x C whenever A and are separated by C in the undirected graph moral(g(an(a C))), which is the moral graph of the smallest ancestral set containing A C. Proof: use Lemma. the above definition of independence in the Lemma. is the original definition of global Markov property for DAGs it is equivalent to the condition (DG) (defined in the previous slide), but we will not prove this equivalence in the lecture =

26 for any distribution µ(x) that factorizes accroding to a directed acyclic graph (DAG) G, and for any disjoint subsets A,, C V, we can test whether x A x x C using ayes ball algorithm: Markov property 3-6. shade all nodes in. place a ball at each node in A 3. let balls propagate, duplicating itself, following the rules shown on the right (Remark: balls do not interact) 4. if no ball can reach C, then x A x x C not not not A = {4} = {3, 5} C = {6} 3 6

27 for any distribution µ(x) that factorizes accroding to a directed acyclic graph (DAG) G, and for any disjoint subsets A,, C V, we can test whether x A x x C using ayes ball algorithm: Markov property 3-6. shade all nodes in. place a ball at each node in A 3. let balls propagate, duplicating itself, following the rules shown on the right (Remark: balls do not interact) 4. if no ball can reach C, then x A x x C not not not A = {4} = {3, 5} C = {6} 3 6

28 for any distribution µ(x) that factorizes accroding to a directed acyclic graph (DAG) G, and for any disjoint subsets A,, C V, we can test whether x A x x C using ayes ball algorithm: Markov property 3-6. shade all nodes in. place a ball at each node in A 3. let balls propagate, duplicating itself, following the rules shown on the right (Remark: balls do not interact) 4. if no ball can reach C, then x A x x C not not not A = {4} = {3, 5} C = {6} 3 6

29 Markov property 3-6 for any distribution µ(x) that factorizes accroding to a directed acyclic graph (DAG) G, and for any disjoint subsets A,, C V, we can test whether x A x x C using ayes ball algorithm:. shade all nodes in. place a ball at each node in A 3. let balls propagate, duplicating itself, following the rules shown on the right (Remark: balls do not interact) 4. if no ball can reach C, then x A x x C not not not 4 A = {4} 5 7 = {3, 5} C = {6} 3 6

30 Markov property 3-6 for any distribution µ(x) that factorizes accroding to a directed acyclic graph (DAG) G, and for any disjoint subsets A,, C V, we can test whether x A x x C using ayes ball algorithm:. shade all nodes in. place a ball at each node in A 3. let balls propagate, duplicating itself, following the rules shown on the right (Remark: balls do not interact) 4. if no ball can reach C, then x A x x C not not not 4 A = {4} 5 7 = {3, 5} C = {6} 3 6

31 for any distribution µ(x) that factorizes accroding to a directed acyclic graph (DAG) G, and for any disjoint subsets A,, C V, we can test whether x A x x C using ayes ball algorithm: Markov property 3-6. shade all nodes in. place a ball at each node in A 3. let balls propagate, duplicating itself, following the rules shown on the right (Remark: balls do not interact) 4. if no ball can reach C, then x A x x C not not not A = {4} = {3, 5} C = {6} 3 6

32 for any distribution µ(x) that factorizes accroding to a directed acyclic graph (DAG) G, and for any disjoint subsets A,, C V, we can test whether x A x x C using ayes ball algorithm: Markov property 3-6. shade all nodes in. place a ball at each node in A 3. let balls propagate, duplicating itself, following the rules shown on the right (Remark: balls do not interact) 4. if no ball can reach C, then x A x x C not not not A = {4} = {3, 5} C = {6} 3 6

33 for any distribution µ(x) that factorizes accroding to a directed acyclic graph (DAG) G, and for any disjoint subsets A,, C V, we can test whether x A x x C using ayes ball algorithm: Markov property 3-6. shade all nodes in. place a ball at each node in A 3. let balls propagate, duplicating itself, following the rules shown on the right (Remark: balls do not interact) 4. if no ball can reach C, then x A x x C not not not A = {4} = {3, 5} C = {6} 3 6

34 for any distribution µ(x) that factorizes accroding to a directed acyclic graph (DAG) G, and for any disjoint subsets A,, C V, we can test whether x A x x C using ayes ball algorithm: Markov property 3-6. shade all nodes in. place a ball at each node in A 3. let balls propagate, duplicating itself, following the rules shown on the right (Remark: balls do not interact) 4. if no ball can reach C, then x A x x C not not not A = {4} = {3, 5} C = {6} 3 6

35 for any distribution µ(x) that factorizes accroding to a directed acyclic graph (DAG) G, and for any disjoint subsets A,, C V, we can test whether x A x x C using ayes ball algorithm: Markov property 3-6. shade all nodes in. place a ball at each node in A 3. let balls propagate, duplicating itself, following the rules shown on the right (Remark: balls do not interact) 4. if no ball can reach C, then x A x x C not not not balls are destroyed Q.when do we stop? A = {4} = {3, 5} C = {6} implies x 4 {x 3, x 5 } x 6

36 x {x, x 4 } x 3, x {x, x 3 } x 4 Markov property 3-7 if a distribution µ(x) factorizes according to a DAG G, i.e. µ(x) = i V µ i (x i x π(i) ) then µ(x) satisfies all the conditional independencies obtainable by ayes ball if a distribution µ(x) satisfies all the conditional independencies obtainable by ayes ball on a DAG G, then we can find a factorization of µ(x) on G in the worst-case ayes ball has runtime O( E ) [Shachter 998] there are conditional independencies that cannot be represented by a ayesian network, but is represented by a MRF, Example: µ(x) = ψ (x, x )ψ 3 (x, x 3 )ψ 34 (x 3, x 4 )ψ 4 (x 4, x ) 4 3

37 Markov property 3-8 Markov property of factor graphs Factor graphs are more fine grained than undirected graphical models ψ(x, x, x 3 ) ψ (x, x )ψ 3 (x, x 3 )ψ 3 (x 3, x ) ψ 3 (x, x, x 3 ) all three encodes same independencies, but different factorizations (in particular 3 X vs. X 3 ) set of independencies represented by MRF is the same as FG but FG can represent a larger set of factorizations for a factor graph G = (V, F, E), for any disjoint subsets A,, C V, µ(x) satisfy x A x x C whenever separates A and C

38 Markov property 3-9 Independence maps (I-maps) which graph (graphical model) is good? let I(G) denote all conditional independencies implied by a graph G let I(µ) denote all conditional independencies of a distribution µ( ) G is an I-map of µ if µ satisfy all the independencies of the graph G, i.e. I(G) I(µ) given µ(x), can we construct a G that captures as many independencies as possible? G is a minimal I-map for µ(x) if G is an I-map for µ(x), and removing a single edge from G causes the graph to no longer be an I-map

39 Markov property 3-30 Constructing minimal I-maps Constructing minimal I-maps for ayesian network. choose an (arbitrary) ordering of x, x,..., x n. consider a directed graph for ayes rule µ(x) = µ(x )µ(x x ) µ(x n x,..., x n ) 3. for each i, select its parents π(i) to be the minimal subset of {x,..., x i } such that x i x π(i) {x..., x i } \ x π(i) minimal I-map for N is not unique there are n! choices of the ordering, and a priori we cannot tell which ordering is best even for the same ordering, the I-map might not be unique e.g. X = X and X 3 = X + Z, then x 3 x x and x 3 x x, but x 3 (x, x ) so there are two I-maps for the ordering (,,3) = =

40 Markov property 3-3 Constructing minimal I-maps for Markov random fields. consider a complete undirected graph. for each edge (i, j), remove the edge if i and j are conditionally independent given all the rest of nodes if µ(x) > 0, then the resulting graph is the unique minimal I-map of µ can you prove this? idea: when µ > 0, graph obtained from the pairwise independencies is exactly the same as the graph obtained from the global independencies (from Hammersley-Clifford) and hence there is no loss in only considering the pairwise conditional independencies when constructing the graph when µ(x) = 0 for some x, then I-map might not be unique [Homework.]

41 Markov property 3-3 Moralization moralization converts a N D to a MRF G such that I(G) I(D) resulting G is an minimal (undirected) I-map of I(D). retain all edges in D and make them undirected. connect every pair of parents with an edge 3 3 when do we lose nothing in converting a directed graph to an undirected one? (when moralization adds no more edges than already present in D) there exists a G such that I(D) = I(G) if and only if moralization of D does not add any edges proof of : in the example above, the independence is not preserved, and this argument can be made general proof of : we will not prove this in the lecture

42 Perfect maps (P-maps) G is a perfect map (P-map) for µ(x) if I(G) = I(µ) set of all distributions on V Markov chains trees perfect undirected chordal undirected perfect directed examples: trees have efficient inference algorithms, which plays a crucial role in developing efficient algorithms for other graphs as well Markov property 3-33

43 Markov property 3-34 Trees undirected tree is a connected undirected graph with no cycle directed tree is a connected directed graph where each node has at most one parent conversion from MRF on tree to N on tree: take any node as root and start directing edges away from the root non-unique all conversions result in the same set of independencies example: Markov chain, hidden Markov models exact inference is extremely efficient on trees

44 Markov property 3-35 Chordal graphs consider an undirected graph G(V, E) we say (i,..., i k ) form a trail in the graph G = (V, E) if for every j {,..., k }, we have (i j, i j+ ) E a loop is a trail (i,..., i k ) where i k = i an undirected graph is chordal if the longest minimal loop is a triangle given G, if there exists a DAG D such that if I(G) = I(D) then G is a chordal graph proof idea: any loop of size > 3 with no chord (a chord is an edge connecting two vertices that are not consecutive in the loop), have independency that cannot be encoded by a DAG

45 the following are equivalent for MRFs () G is chordal () there exists an orientation of the edges in G that gives a DAG D whose moral graph is G (3) there exists a directed graphical model with conditional independencies identical to those implied by G, i.e. I(G) = I(D) proof of () (3): follows from the moralization slide (we did not provide a proof) proof of (3) (): proved in the previous slide proof of () (): Lemma 3. a chordal G is recursively simplicial. given a recursively simplicial graph G, we can construct a DAG D as follows: start with empty D 0 and fix a simplicial ordering (,..., n) start from node x, sequentially adding one node at a time, adding x t to D t and add edges from x t towards x t (unless there is already an edge in the opposite direction) it is clear for the construction that D is acyclic the moral graph of D has no additional edges, since the parents of any node form a clique in the original graph Markov property 3-36

46 Markov property 3-37 use triangulation to construct such a directed graph D from an undirected chordal graph G, which we will not cover in this course a node in G is simplicial if the subgraph defined by its neighbors form a complete subgraph a graph G is recursively simplicial if it contains a simplicial node x i and when x i is removed from the (sub)graph, what remains is recursively simplicial proof of Lemma 3. is omitted here, but refer to e.g. bartlett/courses/009fallcs8a/graphnotes.pdf

47 Markov property 3-38 Roadmap Cond. Indep. Factorization Graphical Graph Cond. Indep. µ(x) µ(x) Model G implied by G x {x, x 3 } x 4 ; Z ψa (x a ) FG Factor Markov x 4 {} x 7 ; Z ψc (x C ) MRF Undirected Markov. ψi (x i x π(i) ) N Directed Markov Undirected graphical models: (F) (G) (L) (P) For any positive µ(x) that satisfy (G), we can find a factorization (F) List of all positive distributions {µ(x)} that factorize according to (F) is equivalent as the list of all distributions that satisfy (G) Directed graphical models: (F) (G) (L) (O)

### Probabilistic Graphical Models

Probabilistic Graphical Models Raquel Urtasun and Tamir Hazan TTI Chicago April 4, 2011 Raquel Urtasun and Tamir Hazan (TTI-C) Graphical Models April 4, 2011 1 / 22 Bayesian Networks and independences

### 5 Directed acyclic graphs

5 Directed acyclic graphs (5.1) Introduction In many statistical studies we have prior knowledge about a temporal or causal ordering of the variables. In this chapter we will use directed graphs to incorporate

### The Basics of Graphical Models

The Basics of Graphical Models David M. Blei Columbia University October 3, 2015 Introduction These notes follow Chapter 2 of An Introduction to Probabilistic Graphical Models by Michael Jordan. Many figures

### 3. The Junction Tree Algorithms

A Short Course on Graphical Models 3. The Junction Tree Algorithms Mark Paskin mark@paskin.org 1 Review: conditional independence Two random variables X and Y are independent (written X Y ) iff p X ( )

### Distributed Computing over Communication Networks: Maximal Independent Set

Distributed Computing over Communication Networks: Maximal Independent Set What is a MIS? MIS An independent set (IS) of an undirected graph is a subset U of nodes such that no two nodes in U are adjacent.

### 2.3 Scheduling jobs on identical parallel machines

2.3 Scheduling jobs on identical parallel machines There are jobs to be processed, and there are identical machines (running in parallel) to which each job may be assigned Each job = 1,,, must be processed

### Exponential time algorithms for graph coloring

Exponential time algorithms for graph coloring Uriel Feige Lecture notes, March 14, 2011 1 Introduction Let [n] denote the set {1,..., k}. A k-labeling of vertices of a graph G(V, E) is a function V [k].

### Handout #Ch7 San Skulrattanakulchai Gustavus Adolphus College Dec 6, 2010. Chapter 7: Digraphs

MCS-236: Graph Theory Handout #Ch7 San Skulrattanakulchai Gustavus Adolphus College Dec 6, 2010 Chapter 7: Digraphs Strong Digraphs Definitions. A digraph is an ordered pair (V, E), where V is the set

### Examples and Proofs of Inference in Junction Trees

Examples and Proofs of Inference in Junction Trees Peter Lucas LIAC, Leiden University February 4, 2016 1 Representation and notation Let P(V) be a joint probability distribution, where V stands for a

### Finding and counting given length cycles

Finding and counting given length cycles Noga Alon Raphael Yuster Uri Zwick Abstract We present an assortment of methods for finding and counting simple cycles of a given length in directed and undirected

### A 2-factor in which each cycle has long length in claw-free graphs

A -factor in which each cycle has long length in claw-free graphs Roman Čada Shuya Chiba Kiyoshi Yoshimoto 3 Department of Mathematics University of West Bohemia and Institute of Theoretical Computer Science

### Triangle deletion. Ernie Croot. February 3, 2010

Triangle deletion Ernie Croot February 3, 2010 1 Introduction The purpose of this note is to give an intuitive outline of the triangle deletion theorem of Ruzsa and Szemerédi, which says that if G = (V,

### Competition Parameters of a Graph

AKCE J. Graphs. Combin., 4, No. 2 (2007), pp. 183-190 Competition Parameters of a Graph Suk J. Seo Computer Science Department Middle Tennessee State University Murfreesboro, TN 37132, U.S.A E-mail: sseo@mtsu.edu

### Tree-representation of set families and applications to combinatorial decompositions

Tree-representation of set families and applications to combinatorial decompositions Binh-Minh Bui-Xuan a, Michel Habib b Michaël Rao c a Department of Informatics, University of Bergen, Norway. buixuan@ii.uib.no

### Lecture 15 An Arithmetic Circuit Lowerbound and Flows in Graphs

CSE599s: Extremal Combinatorics November 21, 2011 Lecture 15 An Arithmetic Circuit Lowerbound and Flows in Graphs Lecturer: Anup Rao 1 An Arithmetic Circuit Lower Bound An arithmetic circuit is just like

### On the k-path cover problem for cacti

On the k-path cover problem for cacti Zemin Jin and Xueliang Li Center for Combinatorics and LPMC Nankai University Tianjin 300071, P.R. China zeminjin@eyou.com, x.li@eyou.com Abstract In this paper we

### Connectivity and cuts

Math 104, Graph Theory February 19, 2013 Measure of connectivity How connected are each of these graphs? > increasing connectivity > I G 1 is a tree, so it is a connected graph w/minimum # of edges. Every

### Lecture 1: Course overview, circuits, and formulas

Lecture 1: Course overview, circuits, and formulas Topics in Complexity Theory and Pseudorandomness (Spring 2013) Rutgers University Swastik Kopparty Scribes: John Kim, Ben Lund 1 Course Information Swastik

### GRAPH THEORY LECTURE 4: TREES

GRAPH THEORY LECTURE 4: TREES Abstract. 3.1 presents some standard characterizations and properties of trees. 3.2 presents several different types of trees. 3.7 develops a counting method based on a bijection

### Outline. NP-completeness. When is a problem easy? When is a problem hard? Today. Euler Circuits

Outline NP-completeness Examples of Easy vs. Hard problems Euler circuit vs. Hamiltonian circuit Shortest Path vs. Longest Path 2-pairs sum vs. general Subset Sum Reducing one problem to another Clique

### Analysis of Algorithms, I

Analysis of Algorithms, I CSOR W4231.002 Eleni Drinea Computer Science Department Columbia University Thursday, February 26, 2015 Outline 1 Recap 2 Representing graphs 3 Breadth-first search (BFS) 4 Applications

### Social Media Mining. Graph Essentials

Graph Essentials Graph Basics Measures Graph and Essentials Metrics 2 2 Nodes and Edges A network is a graph nodes, actors, or vertices (plural of vertex) Connections, edges or ties Edge Node Measures

### 5.1 Bipartite Matching

CS787: Advanced Algorithms Lecture 5: Applications of Network Flow In the last lecture, we looked at the problem of finding the maximum flow in a graph, and how it can be efficiently solved using the Ford-Fulkerson

### Small Maximal Independent Sets and Faster Exact Graph Coloring

Small Maximal Independent Sets and Faster Exact Graph Coloring David Eppstein Univ. of California, Irvine Dept. of Information and Computer Science The Exact Graph Coloring Problem: Given an undirected

### COMBINATORIAL PROPERTIES OF THE HIGMAN-SIMS GRAPH. 1. Introduction

COMBINATORIAL PROPERTIES OF THE HIGMAN-SIMS GRAPH ZACHARY ABEL 1. Introduction In this survey we discuss properties of the Higman-Sims graph, which has 100 vertices, 1100 edges, and is 22 regular. In fact

### POLYNOMIAL RINGS AND UNIQUE FACTORIZATION DOMAINS

POLYNOMIAL RINGS AND UNIQUE FACTORIZATION DOMAINS RUSS WOODROOFE 1. Unique Factorization Domains Throughout the following, we think of R as sitting inside R[x] as the constant polynomials (of degree 0).

### Problem Set 7 Solutions

8 8 Introduction to Algorithms May 7, 2004 Massachusetts Institute of Technology 6.046J/18.410J Professors Erik Demaine and Shafi Goldwasser Handout 25 Problem Set 7 Solutions This problem set is due in

### IE 680 Special Topics in Production Systems: Networks, Routing and Logistics*

IE 680 Special Topics in Production Systems: Networks, Routing and Logistics* Rakesh Nagi Department of Industrial Engineering University at Buffalo (SUNY) *Lecture notes from Network Flows by Ahuja, Magnanti

### Discrete Mathematics & Mathematical Reasoning Chapter 10: Graphs

Discrete Mathematics & Mathematical Reasoning Chapter 10: Graphs Kousha Etessami U. of Edinburgh, UK Kousha Etessami (U. of Edinburgh, UK) Discrete Mathematics (Chapter 6) 1 / 13 Overview Graphs and Graph

### Divisor graphs have arbitrary order and size

Divisor graphs have arbitrary order and size arxiv:math/0606483v1 [math.co] 20 Jun 2006 Le Anh Vinh School of Mathematics University of New South Wales Sydney 2052 Australia Abstract A divisor graph G

### Cost Model: Work, Span and Parallelism. 1 The RAM model for sequential computation:

CSE341T 08/31/2015 Lecture 3 Cost Model: Work, Span and Parallelism In this lecture, we will look at how one analyze a parallel program written using Cilk Plus. When we analyze the cost of an algorithm

### Outline 2.1 Graph Isomorphism 2.2 Automorphisms and Symmetry 2.3 Subgraphs, part 1

GRAPH THEORY LECTURE STRUCTURE AND REPRESENTATION PART A Abstract. Chapter focuses on the question of when two graphs are to be regarded as the same, on symmetries, and on subgraphs.. discusses the concept

### Scheduling Shop Scheduling. Tim Nieberg

Scheduling Shop Scheduling Tim Nieberg Shop models: General Introduction Remark: Consider non preemptive problems with regular objectives Notation Shop Problems: m machines, n jobs 1,..., n operations

### Definition 11.1. Given a graph G on n vertices, we define the following quantities:

Lecture 11 The Lovász ϑ Function 11.1 Perfect graphs We begin with some background on perfect graphs. graphs. First, we define some quantities on Definition 11.1. Given a graph G on n vertices, we define

### Midterm Practice Problems

6.042/8.062J Mathematics for Computer Science October 2, 200 Tom Leighton, Marten van Dijk, and Brooke Cowan Midterm Practice Problems Problem. [0 points] In problem set you showed that the nand operator

### Graphical degree sequences and realizations

swap Graphical and realizations Péter L. Erdös Alfréd Rényi Institute of Mathematics Hungarian Academy of Sciences MAPCON 12 MPIPKS - Dresden, May 15, 2012 swap Graphical and realizations Péter L. Erdös

### Resource Allocation with Time Intervals

Resource Allocation with Time Intervals Andreas Darmann Ulrich Pferschy Joachim Schauer Abstract We study a resource allocation problem where jobs have the following characteristics: Each job consumes

### Large induced subgraphs with all degrees odd

Large induced subgraphs with all degrees odd A.D. Scott Department of Pure Mathematics and Mathematical Statistics, University of Cambridge, England Abstract: We prove that every connected graph of order

### Cycles and clique-minors in expanders

Cycles and clique-minors in expanders Benny Sudakov UCLA and Princeton University Expanders Definition: The vertex boundary of a subset X of a graph G: X = { all vertices in G\X with at least one neighbor

### Markov random fields and Gibbs measures

Chapter Markov random fields and Gibbs measures 1. Conditional independence Suppose X i is a random element of (X i, B i ), for i = 1, 2, 3, with all X i defined on the same probability space (.F, P).

### Lecture 11: Graphical Models for Inference

Lecture 11: Graphical Models for Inference So far we have seen two graphical models that are used for inference - the Bayesian network and the Join tree. These two both represent the same joint probability

### The degree, size and chromatic index of a uniform hypergraph

The degree, size and chromatic index of a uniform hypergraph Noga Alon Jeong Han Kim Abstract Let H be a k-uniform hypergraph in which no two edges share more than t common vertices, and let D denote the

### Labeling outerplanar graphs with maximum degree three

Labeling outerplanar graphs with maximum degree three Xiangwen Li 1 and Sanming Zhou 2 1 Department of Mathematics Huazhong Normal University, Wuhan 430079, China 2 Department of Mathematics and Statistics

### Graph Theory Problems and Solutions

raph Theory Problems and Solutions Tom Davis tomrdavis@earthlink.net http://www.geometer.org/mathcircles November, 005 Problems. Prove that the sum of the degrees of the vertices of any finite graph is

### Homework Exam 1, Geometric Algorithms, 2016

Homework Exam 1, Geometric Algorithms, 2016 1. (3 points) Let P be a convex polyhedron in 3-dimensional space. The boundary of P is represented as a DCEL, storing the incidence relationships between the

### CS 598CSC: Combinatorial Optimization Lecture date: 2/4/2010

CS 598CSC: Combinatorial Optimization Lecture date: /4/010 Instructor: Chandra Chekuri Scribe: David Morrison Gomory-Hu Trees (The work in this section closely follows [3]) Let G = (V, E) be an undirected

### The Joint Probability Distribution (JPD) of a set of n binary variables involve a huge number of parameters

DEFINING PROILISTI MODELS The Joint Probability Distribution (JPD) of a set of n binary variables involve a huge number of parameters 2 n (larger than 10 25 for only 100 variables). x y z p(x, y, z) 0

### Full and Complete Binary Trees

Full and Complete Binary Trees Binary Tree Theorems 1 Here are two important types of binary trees. Note that the definitions, while similar, are logically independent. Definition: a binary tree T is full

### 1 Definitions. Supplementary Material for: Digraphs. Concept graphs

Supplementary Material for: van Rooij, I., Evans, P., Müller, M., Gedge, J. & Wareham, T. (2008). Identifying Sources of Intractability in Cognitive Models: An Illustration using Analogical Structure Mapping.

### SCAN: A Structural Clustering Algorithm for Networks

SCAN: A Structural Clustering Algorithm for Networks Xiaowei Xu, Nurcan Yuruk, Zhidan Feng (University of Arkansas at Little Rock) Thomas A. J. Schweiger (Acxiom Corporation) Networks scaling: #edges connected

### Mean Ramsey-Turán numbers

Mean Ramsey-Turán numbers Raphael Yuster Department of Mathematics University of Haifa at Oranim Tivon 36006, Israel Abstract A ρ-mean coloring of a graph is a coloring of the edges such that the average

### INDISTINGUISHABILITY OF ABSOLUTELY CONTINUOUS AND SINGULAR DISTRIBUTIONS

INDISTINGUISHABILITY OF ABSOLUTELY CONTINUOUS AND SINGULAR DISTRIBUTIONS STEVEN P. LALLEY AND ANDREW NOBEL Abstract. It is shown that there are no consistent decision rules for the hypothesis testing problem

### Graphs without proper subgraphs of minimum degree 3 and short cycles

Graphs without proper subgraphs of minimum degree 3 and short cycles Lothar Narins, Alexey Pokrovskiy, Tibor Szabó Department of Mathematics, Freie Universität, Berlin, Germany. August 22, 2014 Abstract

### UPPER BOUNDS ON THE L(2, 1)-LABELING NUMBER OF GRAPHS WITH MAXIMUM DEGREE

UPPER BOUNDS ON THE L(2, 1)-LABELING NUMBER OF GRAPHS WITH MAXIMUM DEGREE ANDREW LUM ADVISOR: DAVID GUICHARD ABSTRACT. L(2,1)-labeling was first defined by Jerrold Griggs [Gr, 1992] as a way to use graphs

### Disjoint Compatible Geometric Matchings

Disjoint Compatible Geometric Matchings Mashhood Ishaque Diane L. Souvaine Csaba D. Tóth Abstract We prove that for every even set of n pairwise disjoint line segments in the plane in general position,

### SHARP BOUNDS FOR THE SUM OF THE SQUARES OF THE DEGREES OF A GRAPH

31 Kragujevac J. Math. 25 (2003) 31 49. SHARP BOUNDS FOR THE SUM OF THE SQUARES OF THE DEGREES OF A GRAPH Kinkar Ch. Das Department of Mathematics, Indian Institute of Technology, Kharagpur 721302, W.B.,

### Discrete Mathematics Problems

Discrete Mathematics Problems William F. Klostermeyer School of Computing University of North Florida Jacksonville, FL 32224 E-mail: wkloster@unf.edu Contents 0 Preface 3 1 Logic 5 1.1 Basics...............................

### Approximation Algorithms

Approximation Algorithms or: How I Learned to Stop Worrying and Deal with NP-Completeness Ong Jit Sheng, Jonathan (A0073924B) March, 2012 Overview Key Results (I) General techniques: Greedy algorithms

### On Integer Additive Set-Indexers of Graphs

On Integer Additive Set-Indexers of Graphs arxiv:1312.7672v4 [math.co] 2 Mar 2014 N K Sudev and K A Germina Abstract A set-indexer of a graph G is an injective set-valued function f : V (G) 2 X such that

### A first step towards modeling semistructured data in hybrid multimodal logic

A first step towards modeling semistructured data in hybrid multimodal logic Nicole Bidoit * Serenella Cerrito ** Virginie Thion * * LRI UMR CNRS 8623, Université Paris 11, Centre d Orsay. ** LaMI UMR

### 3. Equivalence Relations. Discussion

3. EQUIVALENCE RELATIONS 33 3. Equivalence Relations 3.1. Definition of an Equivalence Relations. Definition 3.1.1. A relation R on a set A is an equivalence relation if and only if R is reflexive, symmetric,

### INCIDENCE-BETWEENNESS GEOMETRY

INCIDENCE-BETWEENNESS GEOMETRY MATH 410, CSUSM. SPRING 2008. PROFESSOR AITKEN This document covers the geometry that can be developed with just the axioms related to incidence and betweenness. The full

### Lecture 17 : Equivalence and Order Relations DRAFT

CS/Math 240: Introduction to Discrete Mathematics 3/31/2011 Lecture 17 : Equivalence and Order Relations Instructor: Dieter van Melkebeek Scribe: Dalibor Zelený DRAFT Last lecture we introduced the notion

### Part 2: Community Detection

Chapter 8: Graph Data Part 2: Community Detection Based on Leskovec, Rajaraman, Ullman 2014: Mining of Massive Datasets Big Data Management and Analytics Outline Community Detection - Social networks -

### TOPIC 3: CONTINUITY OF FUNCTIONS

TOPIC 3: CONTINUITY OF FUNCTIONS. Absolute value We work in the field of real numbers, R. For the study of the properties of functions we need the concept of absolute value of a number. Definition.. Let

### 8. Matchings and Factors

8. Matchings and Factors Consider the formation of an executive council by the parliament committee. Each committee needs to designate one of its members as an official representative to sit on the council,

### SEQUENCES OF MAXIMAL DEGREE VERTICES IN GRAPHS. Nickolay Khadzhiivanov, Nedyalko Nenov

Serdica Math. J. 30 (2004), 95 102 SEQUENCES OF MAXIMAL DEGREE VERTICES IN GRAPHS Nickolay Khadzhiivanov, Nedyalko Nenov Communicated by V. Drensky Abstract. Let Γ(M) where M V (G) be the set of all vertices

### Spring 2007 Math 510 Hints for practice problems

Spring 2007 Math 510 Hints for practice problems Section 1 Imagine a prison consisting of 4 cells arranged like the squares of an -chessboard There are doors between all adjacent cells A prisoner in one

### 1 if 1 x 0 1 if 0 x 1

Chapter 3 Continuity In this chapter we begin by defining the fundamental notion of continuity for real valued functions of a single real variable. When trying to decide whether a given function is or

V. Adamchik 1 Graph Theory Victor Adamchik Fall of 2005 Plan 1. Basic Vocabulary 2. Regular graph 3. Connectivity 4. Representing Graphs Introduction A.Aho and J.Ulman acknowledge that Fundamentally, computer

### Class notes Program Analysis course given by Prof. Mooly Sagiv Computer Science Department, Tel Aviv University second lecture 8/3/2007

Constant Propagation Class notes Program Analysis course given by Prof. Mooly Sagiv Computer Science Department, Tel Aviv University second lecture 8/3/2007 Osnat Minz and Mati Shomrat Introduction This

### The Union-Find Problem Kruskal s algorithm for finding an MST presented us with a problem in data-structure design. As we looked at each edge,

The Union-Find Problem Kruskal s algorithm for finding an MST presented us with a problem in data-structure design. As we looked at each edge, cheapest first, we had to determine whether its two endpoints

### The positive minimum degree game on sparse graphs

The positive minimum degree game on sparse graphs József Balogh Department of Mathematical Sciences University of Illinois, USA jobal@math.uiuc.edu András Pluhár Department of Computer Science University

### Generalized Induced Factor Problems

Egerváry Research Group on Combinatorial Optimization Technical reports TR-2002-07. Published by the Egrerváry Research Group, Pázmány P. sétány 1/C, H 1117, Budapest, Hungary. Web site: www.cs.elte.hu/egres.

### A Sublinear Bipartiteness Tester for Bounded Degree Graphs

A Sublinear Bipartiteness Tester for Bounded Degree Graphs Oded Goldreich Dana Ron February 5, 1998 Abstract We present a sublinear-time algorithm for testing whether a bounded degree graph is bipartite

### ON INDUCED SUBGRAPHS WITH ALL DEGREES ODD. 1. Introduction

ON INDUCED SUBGRAPHS WITH ALL DEGREES ODD A.D. SCOTT Abstract. Gallai proved that the vertex set of any graph can be partitioned into two sets, each inducing a subgraph with all degrees even. We prove

### Tree sums and maximal connected I-spaces

Tree sums and maximal connected I-spaces Adam Bartoš drekin@gmail.com Faculty of Mathematics and Physics Charles University in Prague Twelfth Symposium on General Topology Prague, July 2016 Maximal and

### Data Structures in Java. Session 16 Instructor: Bert Huang

Data Structures in Java Session 16 Instructor: Bert Huang http://www.cs.columbia.edu/~bert/courses/3134 Announcements Homework 4 due next class Remaining grades: hw4, hw5, hw6 25% Final exam 30% Midterm

### Introduction to Logic in Computer Science: Autumn 2006

Introduction to Logic in Computer Science: Autumn 2006 Ulle Endriss Institute for Logic, Language and Computation University of Amsterdam Ulle Endriss 1 Plan for Today Now that we have a basic understanding

### Analysis of Algorithms I: Binary Search Trees

Analysis of Algorithms I: Binary Search Trees Xi Chen Columbia University Hash table: A data structure that maintains a subset of keys from a universe set U = {0, 1,..., p 1} and supports all three dictionary

### Topology-based network security

Topology-based network security Tiit Pikma Supervised by Vitaly Skachek Research Seminar in Cryptography University of Tartu, Spring 2013 1 Introduction In both wired and wireless networks, there is the

### 15-750 Graduate Algorithms Spring 2010

15-750 Graduate Algorithms Spring 2010 1 Cheap Cut (25 pts.) Suppose we are given a directed network G = (V, E) with a root node r and a set of terminal nodes T V. We would like to remove the fewest number

### OHJ-2306 Introduction to Theoretical Computer Science, Fall 2012 8.11.2012

276 The P vs. NP problem is a major unsolved problem in computer science It is one of the seven Millennium Prize Problems selected by the Clay Mathematics Institute to carry a \$ 1,000,000 prize for the

### Lecture Note 1 Set and Probability Theory. MIT 14.30 Spring 2006 Herman Bennett

Lecture Note 1 Set and Probability Theory MIT 14.30 Spring 2006 Herman Bennett 1 Set Theory 1.1 Definitions and Theorems 1. Experiment: any action or process whose outcome is subject to uncertainty. 2.

### arxiv:1202.5749v1 [cs.ds] 26 Feb 2012

Fixed-parameter tractability of multicut in directed acyclic graphs Stefan Kratsch Marcin Pilipczuk Micha l Pilipczuk Magnus Wahlström arxiv:1202.5749v1 [cs.ds] 26 Feb 2012 Abstract The Multicut problem,

### On an anti-ramsey type result

On an anti-ramsey type result Noga Alon, Hanno Lefmann and Vojtĕch Rödl Abstract We consider anti-ramsey type results. For a given coloring of the k-element subsets of an n-element set X, where two k-element

### I. GROUPS: BASIC DEFINITIONS AND EXAMPLES

I GROUPS: BASIC DEFINITIONS AND EXAMPLES Definition 1: An operation on a set G is a function : G G G Definition 2: A group is a set G which is equipped with an operation and a special element e G, called

### CMPS 102 Solutions to Homework 1

CMPS 0 Solutions to Homework Lindsay Brown, lbrown@soe.ucsc.edu September 9, 005 Problem..- p. 3 For inputs of size n insertion sort runs in 8n steps, while merge sort runs in 64n lg n steps. For which

### SOLUTIONS TO ASSIGNMENT 1 MATH 576

SOLUTIONS TO ASSIGNMENT 1 MATH 576 SOLUTIONS BY OLIVIER MARTIN 13 #5. Let T be the topology generated by A on X. We want to show T = J B J where B is the set of all topologies J on X with A J. This amounts

### High degree graphs contain large-star factors

High degree graphs contain large-star factors Dedicated to László Lovász, for his 60th birthday Noga Alon Nicholas Wormald Abstract We show that any finite simple graph with minimum degree d contains a

### 136 CHAPTER 4. INDUCTION, GRAPHS AND TREES

136 TER 4. INDUCTION, GRHS ND TREES 4.3 Graphs In this chapter we introduce a fundamental structural idea of discrete mathematics, that of a graph. Many situations in the applications of discrete mathematics

### Degree Hypergroupoids Associated with Hypergraphs

Filomat 8:1 (014), 119 19 DOI 10.98/FIL1401119F Published by Faculty of Sciences and Mathematics, University of Niš, Serbia Available at: http://www.pmf.ni.ac.rs/filomat Degree Hypergroupoids Associated

### each college c i C has a capacity q i - the maximum number of students it will admit

n colleges in a set C, m applicants in a set A, where m is much larger than n. each college c i C has a capacity q i - the maximum number of students it will admit each college c i has a strict order i

### Every tree contains a large induced subgraph with all degrees odd

Every tree contains a large induced subgraph with all degrees odd A.J. Radcliffe Carnegie Mellon University, Pittsburgh, PA A.D. Scott Department of Pure Mathematics and Mathematical Statistics University

### 6.852: Distributed Algorithms Fall, 2009. Class 2

.8: Distributed Algorithms Fall, 009 Class Today s plan Leader election in a synchronous ring: Lower bound for comparison-based algorithms. Basic computation in general synchronous networks: Leader election

### Uniform Multicommodity Flow through the Complete Graph with Random Edge-capacities

Combinatorics, Probability and Computing (2005) 00, 000 000. c 2005 Cambridge University Press DOI: 10.1017/S0000000000000000 Printed in the United Kingdom Uniform Multicommodity Flow through the Complete

### Lecture 22: November 10

CS271 Randomness & Computation Fall 2011 Lecture 22: November 10 Lecturer: Alistair Sinclair Based on scribe notes by Rafael Frongillo Disclaimer: These notes have not been subjected to the usual scrutiny

### OPTIMAL DESIGN OF DISTRIBUTED SENSOR NETWORKS FOR FIELD RECONSTRUCTION

OPTIMAL DESIGN OF DISTRIBUTED SENSOR NETWORKS FOR FIELD RECONSTRUCTION Sérgio Pequito, Stephen Kruzick, Soummya Kar, José M. F. Moura, A. Pedro Aguiar Department of Electrical and Computer Engineering