# 5 Directed acyclic graphs

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 5 Directed acyclic graphs (5.1) Introduction In many statistical studies we have prior knowledge about a temporal or causal ordering of the variables. In this chapter we will use directed graphs to incorporate such knowledge into a graphical model for the variables. Let X V = (X v ) be a vector of real-valued random variables with probability distribution P and density p. Then the density p can always be decomposed into a product of conditional densities, p(x) = p(x d x 1,..., x d 1 )p(x 1,..., x d 1 ) =... = d p(x v x 1,..., x v 1 ). (5.1) Note that this can be achieved for any ordering of the variables. Now suppose that the conditional density of some variable X v does not depend on all its predecessors, namely X 1,..., X v 1, but only on a subset X Uv, that is, X v is conditionally independent of its predecessors given X Uv. Substituting p(x v x Uv ) for p(x v x 1,..., x v 1 ) in the product (5.1), we obtain v=1 p(x) = d p(x v x Uv ). (5.2) v=1 This recursive dependence structure can be represented by a directed graph G by drawing an arrow from each vertex in U v to v. As an immediate consequence of the recursive factorization, the resulting graph is acyclic, that is, it does not contain any loops. On the other hand, P factorizes with respect to the undirected graph G m which is given by the class of complete subsets D = {{v} U v v V }. This graph can be obtained from the directed graph G by completing all sets {v} U v and then converting all directed edges into undirected ones. The graph G m is called the moral graph of G since it is obtained by marrying all parents of a joint child. As an example, suppose that we want to describe the distribution of a genetic phenotype (such as blood group) in a family. In general, we can assume that the phenotype of the father (X) and that of the mother (Y ) are independent, whereas the phenotype of the child (Z) depend on the phenotypes of both parents. Thus the joint density can be written as p(x, y, z) = p(z x, y)p(y)p(x). The dependencies are represented by the directed graph G in Figure 5.1 (a). The absence of an edge between X and Y indicates that these variables are independent. (a) X Z Y (b) X Z Y Figure 5.1: (a) Directed graph G representing the dependences between some phenotype of father (X), mother (Y ), and child (Z); (b) moral graph G m (b). 36

2 On the other hand, if we know the phenotype of the child then X and Y are no longer dependent. In the case of blood groups, for example, if the child has blood group AB and the father s blood group is A, then the blood group of the mother must be either B or AB. This conditional dependence of X and Y given Z is reflected by an edge in the moral graph G m, which is complete and thus does not encode any conditional independence relations among the variables. (5.2) Remark If the joint probability distribution has no density, we can still use conditional independences of the type to associate a directed graph with P. X v X 1,..., X v 1 X Uj (5.3) Example (Markov chain) Consider a Markov chain on a discrete state space, i.e. a sequence (X t ) t N of random variables such that P(X t = x t X t 1 = x t 1,..., X 1 = x 1 ) = P(X t = x t X t 1 = x t 1 ). Then the joint probability of X 1,..., X T is given by P(X T = x T,..., X 1 = x 1 ) = T P(X t = x t X t 1 = x t 1 ) P(X 1 = x 1 ). t=2 The corresponding graph is depicted in Figure 5.3. X t 2 X t 1 X t X t+1 Figure 5.2: Directed graph G for a Markov chain X t. (5.4) Example (Regression) Let X 1, X 2, ε be independent N (0, σ 2 ) distributed random variables and suppose that Y = αx 1 + βx 2 + ε. Since X 1 and X 2 are independent the joint density can be written as On the other hand we have p(x 1, x 2, y) = p(y x 1, x 2 )p(x 1 )p(x 2 ). cov(x 1, X 2 Y ) = cov(x 1, X 2 ) cov(x 1, Y )var(y ) 1 cov(y, X 2 ) = which implies together with cov(x, Z Y ) = cov(x, Z) p(x 1, x 2, y) = p(x 1 x 2, y)p(x 2 y)p(y). αβ 1 + α 2 + β 2 σ2 Thus different orderings of the variables can lead to different directed graphs. 37

3 (5.5) Example As a last example, suppose that in a sociological study examining the causes for differences in social status the following four variables have been observed: Social status of individual s parents (S p ), Gender of individual (G), School education of individual (E), Social status of individual (S i ). We assume that gender G and social status of the parents are independent reflecting the fact that gender is determined genetically and no selection has been taken place (this might not be true in certain cultures). Thus the joint density can be factorized recursively with respect to the graph in Figure 5.5. G E S i S p Figure 5.3: Directed graph G for sociological study about causes of social status of individuals. The presence or absence of other edges reflects research hypotheses we might be interested in, as for example G E: Is there a gender specific discrimination in education and does this depend on social class (E G S p )? S p E: Is there a social class effect in education and does this possibly depend on gender (E S p G)? G S i : Is there a gender specific discrimination in society - besides effects of education or social class of parents (S i G E, S p )? S p S i : Is there a discrimination of lower social class people - besides effects of education and gender (S i S p G, E)? In discussing the association between social status of the parents and education it is clear that we do not want to adjust for the social status of the individual as this might create a selection bias (an individual with low social status but with parents of high social status is likely to have had a bad education). To avoid such selection biases the the independence structure should be modelled by a directed graph. (5.6) Directed acyclic graphs Let V be a finite and nonempty set. Then a directed graph G over V is given by an ordered pair (V, E) where the elements in V represent the vertices of G and E {a b a, b V, a b} are the edges of G. If there exists an ordering v 1,..., v d of the vertices which is consistent with the graph G, that is v i v j E implies i < j, then G is called a directed acyclic graph (DAG). It is clear that in that case G does not contain any cycle, i.e. a path of the 38

4 form v... v. The vertices pr(v j ) = {v 1,..., v j 1 } are the predecessors of v j. The ordering is not uniquely determined by G. Let a b be an edge in G. The vertex a is called a parent of b and b is a child of a. For v V we define ch(v) = {u V v u E}, the set of children of v, pa(v) = {u V u v E}, the set of parents of v, an(v) = {u V u... v E}, the set of ancestors of v, de(v) = {u V v... u E}, the set of descendants of v, nd(v) = V \de(v), the set of nondescendants of v, For A V we define pa(a) = a A pa(a)\a and similarly ch(a), an(a), de(a), and nd(a). Furthermore, we say that A is ancestral if an(a) A and define An(A) = A an(a) as the ancestral set of A. (5.7) Definition Let P be a probability distribution on the sample space X V and G = (V, E) a directed acyclic graph. (DF)P factorizes recursively with respect to G if P has a density p with respect to a product measure µ on X V such that p(x) = p(x v x pa(v) ) (5.8) Relation to separation in undirected graphs Suppose that P factorizes recursively with respect to a directed acyclic graph G = (V, E). The factorization can be rewritten as p(x V ) = φ Av (x) with A v = {v} pa(v) and φ Av (x) = p(x v x pa(v) ). Therefore P also factorizes with respect to any undirected graph in which the sets A v are complete. To illustrate this relation it is sufficient to look only at graphs with three vertices and two edges. The three possible graphs are shown in Figure 5.8. The first graph is associated with the factorization p(x) = p(x 3 x 2 )p(x 2 x 1 )p(x 1 ) = g(x 3, x 2 )h(x 2, x 1 ), which implies that X 3 X 1 X 2. Similarly the second graph corresponds to p(x) = p(x 3 x 2 )p(x 1 x 2 )p(x 2 ) = g (x 3, x 2 )h (x 2, x 1 ), (a) 2 (b) 2 (c) Figure 5.4: Basic configurations for three vertices and one missing edge. 39

5 which leads to the same conditional independence relation as above. graph, however, results from the factorization The third p(x) = p(x 2 x 1, x 3 )p(x 3 )p(x 1 ). Here, the first factor depends on all three variables and consequently X 1 and X 3 are no longer independent if we condition on X 2. In general, two variables X a and X b become dependent when conditioning on a joint child X c and therefore have to be joined ( married ) in any undirected graph describing P. A subgraph induced by two non-adjacent vertices a and b and their common child c is called an immorality. Hence, a directed acyclic graph can be moralized by marrying all parents with a joint child. (5.9) Definition Let G = (V, E) be a directed acyclic graph. The moral graph of G is defined as the undirected graph G m = (V, E m ) obtained from G by completing all immoralities in G and removing directions from the graph, i.e. for two vertices a and b with a < b we have a b E m a b E c V : {a, b} pa(c). (5.10) Proposition Suppose P factorizes recursively with respect to G and A is an ancestral set. Then the marginal distribution P A factorizes recursively with respect to the subgraph G A induced by A. Proof. We have p(x) = p(x v x pa(v) ) = v A p(x v x pa(v) ) \A p(x v x pa(v) ). Since A is ancestral the first factor does not depend on x V \A and the recursive facorization follows by integration over x V \A. (5.11) Corollary Suppose P factorizes recursively with respect to G. Let A, B, and S be disjoint subsets of V. Then A B S [(G An(A B S) ) m ] X A X B X S. Proof. Let U = An(A B S). By the previous proposition P U factorizes recursively with respect to G U which leads to p(x U ) = p(x v x pa(v) ) } {{ } = v U complete in (G U ) m A U: A complete ψ A (x U ), i.e. P U factorizes with respect to (G U ) m. By Proposition 2.5 P satisfies the global Markov property with respect to (G U ) m which implies that X A X B X S whenever A B S in (G U ) m. 40

6 (5.12) Definition (Markov properties for DAGs) Let P be a probability distribution on the sample space X V and G = (V, E) a directed acyclic graph. (DG)P satisfies the global directed Markov property with respect to G if for all disjoint sets A, B, S V A B S [(G an(a B S) ) m ] X A X B X S. (DL)P satisfies the local directed Markov property with respect to G if X v X nd(v) X pa(v). (DO)P satisfies the ordered directed Markov property with respect to G if X v X pr(v) X pa(v). (DP)P satisfies the pairwise directed Markov property with respect to G if for all a, b V with a pr(b) a b / E X a X b X nd(b)\a. (5.13) Theorem Let P be a probability distribution on X V with density p with respect to some product measure µ on X V. Then (DF) (DG) (DL) (DO) (DP). (5.14) Remark If P has a positive and continuous density then for all properties are equivalent. The last three implications are still valid if P does not have a density. Proof of Theorem In Corollary 5.11 we have already shown that (DF) implies (DG). Since the set {v} nd(v) is ancestral we have {v} nd(v)\pa(v) pa(v) [(G {v} nd(v) ) m )] and hence (DL) follows from (DG). Next, noting that pr(v) nd(v) we get X v X nd(v) X pa(v) (C2) X v X pr(v) X pa(v). which proves (DL) (DO). Factorizing p according to the ordering of the vertices we obtain with (DO) p(x) = p(x v x pr(v) ) = p(x v x pa(v) ), i.e. P satisfies (DF ). Finally it suffices to show that (DL) implies (DP). Suppose that a pr(b) and a b / E. Then a / pa(b) and a nd(b). Hence by application of (C2) and (C3). X v X nd(v) X pa(v) X a X b X nd(b)\a 41

7 a b a b y y x x Figure 5.5: Illustration of the d-separation criterion: There are two paths between a and b (dashed lines) which are both blocked by S = {x, y}. For each paths the blocking is due to different intermediate vertices (filled circles). (5.15) D-separation Let π = e 1,..., e n be a path between a and b with intermediate nodes v 1,..., v n 1. Then v j is a collider in π if the adjecent edges in π meet head-to-head, i.e. e j, e j+1 = v j 1 v j v j+1. Otherwise we call v j a noncollider in π. Let S be a subset of V. The path π is blocked by S (or S-blocked) if it has an intermediate node v j such that v j is a collider and v j / An(S) or v j is a noncollider and v j S. A path which is not blocked by S is called S-open. Let A, B, and S be disjoint subsets of V. Then S d-separates A and B if all paths between A and B are S-blocked. We write A d B S [G]. As an example consider the graph in Figure It is sufficient to consider nonselfintersecting paths. There are two such paths between a and b, which are both blocked by S = {x, y}. The first path (left) is blocked by two vertices, a noncollider in S (x) and a collider which is not in S nor has any descendants in S. In the second path (right) y is a noncollider and therefore blocks the path. (5.16) Theorem Let G = (V, E) be a directed acyclic graph. Then A B S [(G An(A B S) ) m ] A d B S [G]. Proof. Suppose that π is an S-bypassing path from a to b in (G An(A B S) ) m. Every edge a b in π such that a and b are not adjacent in G is due to moralization of G An(A B S) and hence a and b have a common child c in An(A B S). Thus replacing a b by a c b (and converting all undirected edges into directed edges in G) we obtain a path π in G which connects a and b and which does not have any noncolliders in S (since π was assumed to be S-bypassing). Let c 1..., c r be the colliders in π which have no descendants in S (i.e. c j An(S)) and which therefore block π. We show by induction on r that an S-open path can be constructed from π. For r = 0 the path is already S-open. Now suppose 42

8 a * a A c c 1 2 c 3 b b * B S Figure 5.6: Construction of a S-open path between A and B from an S-bypassing path (dashed lines) in (G An(A B S) ) m. that π has r blocking colliders and that for paths with r 1 blocking colliders an S-open path can be constructed. If c 1 An(A) then there exists a directed path φ from c 1 to a A which bypasses S (since c 1 / An(S)). Let φ be the reverse path from a to c j and π = π 1, π 2 where π 1 is a path from a to c 1. It follows that π = φ, π 2 is a path from a to b with r 1 blocking colliders. By assumption we can construct an S-open path between A and B from π. If c 1 An(B) then there exists a directed path φ from c 1 to b B which bypasses S. Let π = π 1, π 2 where π 1 is a path from a to c 1. It follows that π = π 2, φ is a path from a to b with r 1 blocking colliders. By assumption we can construct an S-open path between A and B from π. The construction of an S-open path between A and B from an S-bypassing path in (G An(A B S) ) m is illustrated in Figure 5. Conversely let π be an S-open path from A to B in G. Then the set C of all colliders in π is a subset of An(S) and hence π is also a path in G An(A B S). Let v 1,..., v n 1 be the intermediate nodes of π. If v j S then v j C and thus v j 1 v j+1 (E An(A B S) ) m with v j 1, v j+1 / S. Leaving out all colliders in the sequence v 1,..., v n 1 and connecting consecutive nodes we therefore obtain an S-bypassing path in (G An(A B S) ) m. (5.17) Markov equivalence As we have illustrated in Example 5.4 the ordering of the variables can have an effect on the structure of the directed acyclic graph G representing P. On the other hand, two joint probability of two dependent variables X a and X b can be factorized either way leading to a b or a b. Thus the question arises when two DAGs G 1 and G 2 do encode the same set of conditional independence relations? (5.18) Theorem Let G 1 = (V, E 1 ) and G 2 = (V, E 2 ) be two directed acyclic graphs over V. If G 1 and G 2 have the same skeleton (i.e. a and b are adjacent in G 1 if and only if they are adjacent in G 2 ) and the same immoralities (i.e. induced subgraphs of the form a c b) 43

9 G 1 G 2 Figure 5.7: Two Markov equivalent graphs G 1 and G 2. then they are Markov equivalent, that is, for all pairwise disjoint subsets A, B, and S of V A d B S [G 1 ] A d B S [G 2 ]. As an Example consider the graphs in Figure Since both graphs G 1 and G 2 have the same skeleton and the same immorality they are Markov equivalent. 5.1 Recursive graphical models for discrete data Let X V = (X v ) be a vector of discrete valued random variables. A recursive graphical model for X V is given by a directed acyclic grpah G = (V, E) and a set P(G) of distributions that obey the Markov property with respect to G and thus factorize as p(x) = p(x v x pa(v) ). Using this factorization we can rewrite the likelihood function L (p) of X V L (p) [ ] n(i) p(x v x pa(v) ) i I V = p(x v x pa(v) ) n(i) i cl(v) I cl(v) j I V :j cl(v) =i cl(v) = p(x v x pa(v) ) n(i cl(v)) i cl(v) I cl(v) L v (p), where L v (p) are the likelihood functions obtained when sampling the variables in cl(v) with fixed pa(v)-marginals. It follows that the joint likelihood can be maximized by maximizing each factor L v (p) separately. Recall that for a complete graph G and given total count n the maximum likelihood estimate is given by ˆp(i) = n(i)/n. Similarly since the graph G cl(v) is complete and the counts in the pa(v)-marginal are fixed we have ˆp(i v i pa(v) ) = n(i cl(v)) n(i pa(v) ). (5.19) Theorem The maximum likelihood estimator in the recursice graphical model for graph G is given by ˆp(i) = N(i cl(v) ) n(i pa(v) ). 44 as

10 (a) X Y (b) 3 5 Z Figure 5.8: Two directed acyclic graphs G 1 and G 2. (5.20) Example Suppose that G is of the form in Figure 5.1 (a). Then the MLE is of the form ˆp ijk = n ijk n j n i n ij n n. Despite the conditional independence of X and Y the data cannot be reduced beyond the table of counts n ijk itself. (5.21) Example Consider the graph G is Figure 5.1 (b). The corresponding MLE is given by ˆp ijklm = n i n ij n k n j l n jk m. n n i n n j n jk 45

### 3. The Junction Tree Algorithms

A Short Course on Graphical Models 3. The Junction Tree Algorithms Mark Paskin mark@paskin.org 1 Review: conditional independence Two random variables X and Y are independent (written X Y ) iff p X ( )

### INDISTINGUISHABILITY OF ABSOLUTELY CONTINUOUS AND SINGULAR DISTRIBUTIONS

INDISTINGUISHABILITY OF ABSOLUTELY CONTINUOUS AND SINGULAR DISTRIBUTIONS STEVEN P. LALLEY AND ANDREW NOBEL Abstract. It is shown that there are no consistent decision rules for the hypothesis testing problem

### The Basics of Graphical Models

The Basics of Graphical Models David M. Blei Columbia University October 3, 2015 Introduction These notes follow Chapter 2 of An Introduction to Probabilistic Graphical Models by Michael Jordan. Many figures

### THE NUMBER OF GRAPHS AND A RANDOM GRAPH WITH A GIVEN DEGREE SEQUENCE. Alexander Barvinok

THE NUMBER OF GRAPHS AND A RANDOM GRAPH WITH A GIVEN DEGREE SEQUENCE Alexer Barvinok Papers are available at http://www.math.lsa.umich.edu/ barvinok/papers.html This is a joint work with J.A. Hartigan

### Solutions to Exercises 8

Discrete Mathematics Lent 2009 MA210 Solutions to Exercises 8 (1) Suppose that G is a graph in which every vertex has degree at least k, where k 1, and in which every cycle contains at least 4 vertices.

### SEQUENCES OF MAXIMAL DEGREE VERTICES IN GRAPHS. Nickolay Khadzhiivanov, Nedyalko Nenov

Serdica Math. J. 30 (2004), 95 102 SEQUENCES OF MAXIMAL DEGREE VERTICES IN GRAPHS Nickolay Khadzhiivanov, Nedyalko Nenov Communicated by V. Drensky Abstract. Let Γ(M) where M V (G) be the set of all vertices

### The Joint Probability Distribution (JPD) of a set of n binary variables involve a huge number of parameters

DEFINING PROILISTI MODELS The Joint Probability Distribution (JPD) of a set of n binary variables involve a huge number of parameters 2 n (larger than 10 25 for only 100 variables). x y z p(x, y, z) 0

### Probability, Conditional Independence

Probability, Conditional Independence June 19, 2012 Probability, Conditional Independence Probability Sample space Ω of events Each event ω Ω has an associated measure Probability of the event P(ω) Axioms

### COT5405 Analysis of Algorithms Homework 3 Solutions

COT0 Analysis of Algorithms Homework 3 Solutions. Prove or give a counter example: (a) In the textbook, we have two routines for graph traversal - DFS(G) and BFS(G,s) - where G is a graph and s is any

### Probabilistic Graphical Models

Probabilistic Graphical Models Raquel Urtasun and Tamir Hazan TTI Chicago April 4, 2011 Raquel Urtasun and Tamir Hazan (TTI-C) Graphical Models April 4, 2011 1 / 22 Bayesian Networks and independences

### Granger-causality graphs for multivariate time series

Granger-causality graphs for multivariate time series Michael Eichler Universität Heidelberg Abstract In this paper, we discuss the properties of mixed graphs which visualize causal relationships between

### Partitioning edge-coloured complete graphs into monochromatic cycles and paths

arxiv:1205.5492v1 [math.co] 24 May 2012 Partitioning edge-coloured complete graphs into monochromatic cycles and paths Alexey Pokrovskiy Departement of Mathematics, London School of Economics and Political

### Graph Theory Notes. Vadim Lozin. Institute of Mathematics University of Warwick

Graph Theory Notes Vadim Lozin Institute of Mathematics University of Warwick 1 Introduction A graph G = (V, E) consists of two sets V and E. The elements of V are called the vertices and the elements

### On Integer Additive Set-Indexers of Graphs

On Integer Additive Set-Indexers of Graphs arxiv:1312.7672v4 [math.co] 2 Mar 2014 N K Sudev and K A Germina Abstract A set-indexer of a graph G is an injective set-valued function f : V (G) 2 X such that

### COMBINATORIAL PROPERTIES OF THE HIGMAN-SIMS GRAPH. 1. Introduction

COMBINATORIAL PROPERTIES OF THE HIGMAN-SIMS GRAPH ZACHARY ABEL 1. Introduction In this survey we discuss properties of the Higman-Sims graph, which has 100 vertices, 1100 edges, and is 22 regular. In fact

### 4 Basics of Trees. Petr Hliněný, FI MU Brno 1 FI: MA010: Trees and Forests

4 Basics of Trees Trees, actually acyclic connected simple graphs, are among the simplest graph classes. Despite their simplicity, they still have rich structure and many useful application, such as in

### Discrete Mathematics, Chapter 5: Induction and Recursion

Discrete Mathematics, Chapter 5: Induction and Recursion Richard Mayr University of Edinburgh, UK Richard Mayr (University of Edinburgh, UK) Discrete Mathematics. Chapter 5 1 / 20 Outline 1 Well-founded

### Lecture 17 : Equivalence and Order Relations DRAFT

CS/Math 240: Introduction to Discrete Mathematics 3/31/2011 Lecture 17 : Equivalence and Order Relations Instructor: Dieter van Melkebeek Scribe: Dalibor Zelený DRAFT Last lecture we introduced the notion

### Markov random fields and Gibbs measures

Chapter Markov random fields and Gibbs measures 1. Conditional independence Suppose X i is a random element of (X i, B i ), for i = 1, 2, 3, with all X i defined on the same probability space (.F, P).

### CS 598CSC: Combinatorial Optimization Lecture date: 2/4/2010

CS 598CSC: Combinatorial Optimization Lecture date: /4/010 Instructor: Chandra Chekuri Scribe: David Morrison Gomory-Hu Trees (The work in this section closely follows [3]) Let G = (V, E) be an undirected

### DO NOT RE-DISTRIBUTE THIS SOLUTION FILE

Professor Kindred Math 04 Graph Theory Homework 7 Solutions April 3, 03 Introduction to Graph Theory, West Section 5. 0, variation of 5, 39 Section 5. 9 Section 5.3 3, 8, 3 Section 7. Problems you should

### Section 3 Sequences and Limits, Continued.

Section 3 Sequences and Limits, Continued. Lemma 3.6 Let {a n } n N be a convergent sequence for which a n 0 for all n N and it α 0. Then there exists N N such that for all n N. α a n 3 α In particular

### Theorem A graph T is a tree if, and only if, every two distinct vertices of T are joined by a unique path.

Chapter 3 Trees Section 3. Fundamental Properties of Trees Suppose your city is planning to construct a rapid rail system. They want to construct the most economical system possible that will meet the

### Inner Product Spaces

Math 571 Inner Product Spaces 1. Preliminaries An inner product space is a vector space V along with a function, called an inner product which associates each pair of vectors u, v with a scalar u, v, and

### Connectivity and cuts

Math 104, Graph Theory February 19, 2013 Measure of connectivity How connected are each of these graphs? > increasing connectivity > I G 1 is a tree, so it is a connected graph w/minimum # of edges. Every

### Discrete Mathematics & Mathematical Reasoning Chapter 10: Graphs

Discrete Mathematics & Mathematical Reasoning Chapter 10: Graphs Kousha Etessami U. of Edinburgh, UK Kousha Etessami (U. of Edinburgh, UK) Discrete Mathematics (Chapter 6) 1 / 13 Overview Graphs and Graph

### Degree Hypergroupoids Associated with Hypergraphs

Filomat 8:1 (014), 119 19 DOI 10.98/FIL1401119F Published by Faculty of Sciences and Mathematics, University of Niš, Serbia Available at: http://www.pmf.ni.ac.rs/filomat Degree Hypergroupoids Associated

### Part 2: Community Detection

Chapter 8: Graph Data Part 2: Community Detection Based on Leskovec, Rajaraman, Ullman 2014: Mining of Massive Datasets Big Data Management and Analytics Outline Community Detection - Social networks -

### CMSC 451: Graph Properties, DFS, BFS, etc.

CMSC 451: Graph Properties, DFS, BFS, etc. Slides By: Carl Kingsford Department of Computer Science University of Maryland, College Park Based on Chapter 3 of Algorithm Design by Kleinberg & Tardos. Graphs

### GRAPH THEORY and APPLICATIONS. Trees

GRAPH THEORY and APPLICATIONS Trees Properties Tree: a connected graph with no cycle (acyclic) Forest: a graph with no cycle Paths are trees. Star: A tree consisting of one vertex adjacent to all the others.

### Bayesian vs. Markov Networks

Bayesian vs. Markov Networks Le Song Machine Learning II: dvanced Topics CSE 8803ML, Spring 2012 Conditional Independence ssumptions Local Markov ssumption Global Markov ssumption 𝑋 𝑁𝑜𝑛𝑑𝑒𝑠𝑐𝑒𝑛𝑑𝑎𝑛𝑡𝑋 𝑃𝑎𝑋

### Assignment 7; Due Friday, November 11

Assignment 7; Due Friday, November 9.8 a The set Q is not connected because we can write it as a union of two nonempty disjoint open sets, for instance U = (, 2) and V = ( 2, ). The connected subsets are

### Chapter 4. Trees. 4.1 Basics

Chapter 4 Trees 4.1 Basics A tree is a connected graph with no cycles. A forest is a collection of trees. A vertex of degree one, particularly in a tree, is called a leaf. Trees arise in a variety of applications.

### 6.2 Permutations continued

6.2 Permutations continued Theorem A permutation on a finite set A is either a cycle or can be expressed as a product (composition of disjoint cycles. Proof is by (strong induction on the number, r, of

### A 2-factor in which each cycle has long length in claw-free graphs

A -factor in which each cycle has long length in claw-free graphs Roman Čada Shuya Chiba Kiyoshi Yoshimoto 3 Department of Mathematics University of West Bohemia and Institute of Theoretical Computer Science

### Theorem (The division theorem) Suppose that a and b are integers with b > 0. There exist unique integers q and r so that. a = bq + r and 0 r < b.

Theorem (The division theorem) Suppose that a and b are integers with b > 0. There exist unique integers q and r so that a = bq + r and 0 r < b. We re dividing a by b: q is the quotient and r is the remainder,

### Triangle deletion. Ernie Croot. February 3, 2010

Triangle deletion Ernie Croot February 3, 2010 1 Introduction The purpose of this note is to give an intuitive outline of the triangle deletion theorem of Ruzsa and Szemerédi, which says that if G = (V,

### R u t c o r Research R e p o r t. Boolean functions with a simple certificate for CNF complexity. Ondřej Čepeka Petr Kučera b Petr Savický c

R u t c o r Research R e p o r t Boolean functions with a simple certificate for CNF complexity Ondřej Čepeka Petr Kučera b Petr Savický c RRR 2-2010, January 24, 2010 RUTCOR Rutgers Center for Operations

### Cycles and clique-minors in expanders

Cycles and clique-minors in expanders Benny Sudakov UCLA and Princeton University Expanders Definition: The vertex boundary of a subset X of a graph G: X = { all vertices in G\X with at least one neighbor

### Probabilistic Networks An Introduction to Bayesian Networks and Influence Diagrams

Probabilistic Networks An Introduction to Bayesian Networks and Influence Diagrams Uffe B. Kjærulff Department of Computer Science Aalborg University Anders L. Madsen HUGIN Expert A/S 10 May 2005 2 Contents

### Minimum Spanning Trees

Minimum Spanning Trees Algorithms and 18.304 Presentation Outline 1 Graph Terminology Minimum Spanning Trees 2 3 Outline Graph Terminology Minimum Spanning Trees 1 Graph Terminology Minimum Spanning Trees

### M5MS09. Graphical Modelling

Course: MMS09 Setter: Walden Checker: Ginzberg Editor: Calderhead External: Wood Date: April, 0 MSc EXAMINATIONS (STATISTICS) May-June 0 MMS09 Graphical Modelling Setter s signature Checker s signature

### Relations Graphical View

Relations Slides by Christopher M. Bourke Instructor: Berthe Y. Choueiry Introduction Recall that a relation between elements of two sets is a subset of their Cartesian product (of ordered pairs). A binary

### Planar Tree Transformation: Results and Counterexample

Planar Tree Transformation: Results and Counterexample Selim G Akl, Kamrul Islam, and Henk Meijer School of Computing, Queen s University Kingston, Ontario, Canada K7L 3N6 Abstract We consider the problem

### A simple existence criterion for normal spanning trees in infinite graphs

1 A simple existence criterion for normal spanning trees in infinite graphs Reinhard Diestel Halin proved in 1978 that there exists a normal spanning tree in every connected graph G that satisfies the

### Graphs without proper subgraphs of minimum degree 3 and short cycles

Graphs without proper subgraphs of minimum degree 3 and short cycles Lothar Narins, Alexey Pokrovskiy, Tibor Szabó Department of Mathematics, Freie Universität, Berlin, Germany. August 22, 2014 Abstract

### 1. Relevant standard graph theory

Color identical pairs in 4-chromatic graphs Asbjørn Brændeland I argue that, given a 4-chromatic graph G and a pair of vertices {u, v} in G, if the color of u equals the color of v in every 4-coloring

### P. Jeyanthi and N. Angel Benseera

Opuscula Math. 34, no. 1 (014), 115 1 http://dx.doi.org/10.7494/opmath.014.34.1.115 Opuscula Mathematica A TOTALLY MAGIC CORDIAL LABELING OF ONE-POINT UNION OF n COPIES OF A GRAPH P. Jeyanthi and N. Angel

### S(A) X α for all α Λ. Consequently, S(A) X, by the definition of intersection. Therefore, X is inductive.

MA 274: Exam 2 Study Guide (1) Know the precise definitions of the terms requested for your journal. (2) Review proofs by induction. (3) Be able to prove that something is or isn t an equivalence relation.

### CS5314 Randomized Algorithms. Lecture 16: Balls, Bins, Random Graphs (Random Graphs, Hamiltonian Cycles)

CS5314 Randomized Algorithms Lecture 16: Balls, Bins, Random Graphs (Random Graphs, Hamiltonian Cycles) 1 Objectives Introduce Random Graph Model used to define a probability space for all graphs with

### Forests and Trees: A forest is a graph with no cycles, a tree is a connected forest.

2 Trees What is a tree? Forests and Trees: A forest is a graph with no cycles, a tree is a connected forest. Theorem 2.1 If G is a forest, then comp(g) = V (G) E(G). Proof: We proceed by induction on E(G).

### 4. Joint Distributions of Two Random Variables

4. Joint Distributions of Two Random Variables 4.1 Joint Distributions of Two Discrete Random Variables Suppose the discrete random variables X and Y have supports S X and S Y, respectively. The joint

### Tools for parsimonious edge-colouring of graphs with maximum degree three. J.L. Fouquet and J.M. Vanherpe. Rapport n o RR-2010-10

Tools for parsimonious edge-colouring of graphs with maximum degree three J.L. Fouquet and J.M. Vanherpe LIFO, Université d Orléans Rapport n o RR-2010-10 Tools for parsimonious edge-colouring of graphs

### Discrete Mathematics Problems

Discrete Mathematics Problems William F. Klostermeyer School of Computing University of North Florida Jacksonville, FL 32224 E-mail: wkloster@unf.edu Contents 0 Preface 3 1 Logic 5 1.1 Basics...............................

### Maximum Likelihood Estimation

Math 541: Statistical Theory II Lecturer: Songfeng Zheng Maximum Likelihood Estimation 1 Maximum Likelihood Estimation Maximum likelihood is a relatively simple method of constructing an estimator for

### Large-Sample Learning of Bayesian Networks is NP-Hard

Journal of Machine Learning Research 5 (2004) 1287 1330 Submitted 3/04; Published 10/04 Large-Sample Learning of Bayesian Networks is NP-Hard David Maxwell Chickering David Heckerman Christopher Meek Microsoft

### POWER SETS AND RELATIONS

POWER SETS AND RELATIONS L. MARIZZA A. BAILEY 1. The Power Set Now that we have defined sets as best we can, we can consider a sets of sets. If we were to assume nothing, except the existence of the empty

### Finding and counting given length cycles

Finding and counting given length cycles Noga Alon Raphael Yuster Uri Zwick Abstract We present an assortment of methods for finding and counting simple cycles of a given length in directed and undirected

### Two General Methods to Reduce Delay and Change of Enumeration Algorithms

ISSN 1346-5597 NII Technical Report Two General Methods to Reduce Delay and Change of Enumeration Algorithms Takeaki Uno NII-2003-004E Apr.2003 Two General Methods to Reduce Delay and Change of Enumeration

### Course: Model, Learning, and Inference: Lecture 5

Course: Model, Learning, and Inference: Lecture 5 Alan Yuille Department of Statistics, UCLA Los Angeles, CA 90095 yuille@stat.ucla.edu Abstract Probability distributions on structured representation.

### CSE 20: Discrete Mathematics for Computer Science. Prof. Miles Jones. Today s Topics: Graphs. The Internet graph

Today s Topics: CSE 0: Discrete Mathematics for Computer Science Prof. Miles Jones. Graphs. Some theorems on graphs. Eulerian graphs Graphs! Model relations between pairs of objects The Internet graph!

### 1 Definitions. Supplementary Material for: Digraphs. Concept graphs

Supplementary Material for: van Rooij, I., Evans, P., Müller, M., Gedge, J. & Wareham, T. (2008). Identifying Sources of Intractability in Cognitive Models: An Illustration using Analogical Structure Mapping.

### (Refer Slide Time 1.50)

Discrete Mathematical Structures Dr. Kamala Krithivasan Department of Computer Science and Engineering Indian Institute of Technology, Madras Module -2 Lecture #11 Induction Today we shall consider proof

### PROBLEM SET 7: PIGEON HOLE PRINCIPLE

PROBLEM SET 7: PIGEON HOLE PRINCIPLE The pigeonhole principle is the following observation: Theorem. Suppose that > kn marbles are distributed over n jars, then one jar will contain at least k + marbles.

### Approximation Algorithms

Approximation Algorithms or: How I Learned to Stop Worrying and Deal with NP-Completeness Ong Jit Sheng, Jonathan (A0073924B) March, 2012 Overview Key Results (I) General techniques: Greedy algorithms

### 3. Eulerian and Hamiltonian Graphs

3. Eulerian and Hamiltonian Graphs There are many games and puzzles which can be analysed by graph theoretic concepts. In fact, the two early discoveries which led to the existence of graphs arose from

### 8. Matchings and Factors

8. Matchings and Factors Consider the formation of an executive council by the parliament committee. Each committee needs to designate one of its members as an official representative to sit on the council,

### Lecture 3: Linear methods for classification

Lecture 3: Linear methods for classification Rafael A. Irizarry and Hector Corrada Bravo February, 2010 Today we describe four specific algorithms useful for classification problems: linear regression,

### Notes on Matrix Multiplication and the Transitive Closure

ICS 6D Due: Wednesday, February 25, 2015 Instructor: Sandy Irani Notes on Matrix Multiplication and the Transitive Closure An n m matrix over a set S is an array of elements from S with n rows and m columns.

### (a) (b) (c) Figure 1 : Graphs, multigraphs and digraphs. If the vertices of the leftmost figure are labelled {1, 2, 3, 4} in clockwise order from

4 Graph Theory Throughout these notes, a graph G is a pair (V, E) where V is a set and E is a set of unordered pairs of elements of V. The elements of V are called vertices and the elements of E are called

### Handout #Ch7 San Skulrattanakulchai Gustavus Adolphus College Dec 6, 2010. Chapter 7: Digraphs

MCS-236: Graph Theory Handout #Ch7 San Skulrattanakulchai Gustavus Adolphus College Dec 6, 2010 Chapter 7: Digraphs Strong Digraphs Definitions. A digraph is an ordered pair (V, E), where V is the set

### I. GROUPS: BASIC DEFINITIONS AND EXAMPLES

I GROUPS: BASIC DEFINITIONS AND EXAMPLES Definition 1: An operation on a set G is a function : G G G Definition 2: A group is a set G which is equipped with an operation and a special element e G, called

### Lecture 4: BK inequality 27th August and 6th September, 2007

CSL866: Percolation and Random Graphs IIT Delhi Amitabha Bagchi Scribe: Arindam Pal Lecture 4: BK inequality 27th August and 6th September, 2007 4. Preliminaries The FKG inequality allows us to lower bound

### Lecture 16 : Relations and Functions DRAFT

CS/Math 240: Introduction to Discrete Mathematics 3/29/2011 Lecture 16 : Relations and Functions Instructor: Dieter van Melkebeek Scribe: Dalibor Zelený DRAFT In Lecture 3, we described a correspondence

### Chapter 6 Planarity. Section 6.1 Euler s Formula

Chapter 6 Planarity Section 6.1 Euler s Formula In Chapter 1 we introduced the puzzle of the three houses and the three utilities. The problem was to determine if we could connect each of the three utilities

### Single machine parallel batch scheduling with unbounded capacity

Workshop on Combinatorics and Graph Theory 21th, April, 2006 Nankai University Single machine parallel batch scheduling with unbounded capacity Yuan Jinjiang Department of mathematics, Zhengzhou University

### Reinforcement Learning

Reinforcement Learning LU 2 - Markov Decision Problems and Dynamic Programming Dr. Martin Lauer AG Maschinelles Lernen und Natürlichsprachliche Systeme Albert-Ludwigs-Universität Freiburg martin.lauer@kit.edu

### Principle of Data Reduction

Chapter 6 Principle of Data Reduction 6.1 Introduction An experimenter uses the information in a sample X 1,..., X n to make inferences about an unknown parameter θ. If the sample size n is large, then

### Planarity Planarity

Planarity 8.1 71 Planarity Up until now, graphs have been completely abstract. In Topological Graph Theory, it matters how the graphs are drawn. Do the edges cross? Are there knots in the graph structure?

### 3. Equivalence Relations. Discussion

3. EQUIVALENCE RELATIONS 33 3. Equivalence Relations 3.1. Definition of an Equivalence Relations. Definition 3.1.1. A relation R on a set A is an equivalence relation if and only if R is reflexive, symmetric,

### 1 if 1 x 0 1 if 0 x 1

Chapter 3 Continuity In this chapter we begin by defining the fundamental notion of continuity for real valued functions of a single real variable. When trying to decide whether a given function is or

### 1 Basic Definitions and Concepts in Graph Theory

CME 305: Discrete Mathematics and Algorithms 1 Basic Definitions and Concepts in Graph Theory A graph G(V, E) is a set V of vertices and a set E of edges. In an undirected graph, an edge is an unordered

### CONTINUED FRACTIONS AND PELL S EQUATION. Contents 1. Continued Fractions 1 2. Solution to Pell s Equation 9 References 12

CONTINUED FRACTIONS AND PELL S EQUATION SEUNG HYUN YANG Abstract. In this REU paper, I will use some important characteristics of continued fractions to give the complete set of solutions to Pell s equation.

### 1.3 Induction and Other Proof Techniques

4CHAPTER 1. INTRODUCTORY MATERIAL: SETS, FUNCTIONS AND MATHEMATICAL INDU 1.3 Induction and Other Proof Techniques The purpose of this section is to study the proof technique known as mathematical induction.

### Lecture 15 An Arithmetic Circuit Lowerbound and Flows in Graphs

CSE599s: Extremal Combinatorics November 21, 2011 Lecture 15 An Arithmetic Circuit Lowerbound and Flows in Graphs Lecturer: Anup Rao 1 An Arithmetic Circuit Lower Bound An arithmetic circuit is just like

### E3: PROBABILITY AND STATISTICS lecture notes

E3: PROBABILITY AND STATISTICS lecture notes 2 Contents 1 PROBABILITY THEORY 7 1.1 Experiments and random events............................ 7 1.2 Certain event. Impossible event............................

### Social Media Mining. Network Measures

Klout Measures and Metrics 22 Why Do We Need Measures? Who are the central figures (influential individuals) in the network? What interaction patterns are common in friends? Who are the like-minded users

### Chapter 1. Sigma-Algebras. 1.1 Definition

Chapter 1 Sigma-Algebras 1.1 Definition Consider a set X. A σ algebra F of subsets of X is a collection F of subsets of X satisfying the following conditions: (a) F (b) if B F then its complement B c is

### COLORED GRAPHS AND THEIR PROPERTIES

COLORED GRAPHS AND THEIR PROPERTIES BEN STEVENS 1. Introduction This paper is concerned with the upper bound on the chromatic number for graphs of maximum vertex degree under three different sets of coloring

### The sample space for a pair of die rolls is the set. The sample space for a random number between 0 and 1 is the interval [0, 1].

Probability Theory Probability Spaces and Events Consider a random experiment with several possible outcomes. For example, we might roll a pair of dice, flip a coin three times, or choose a random real

### Sets and functions. {x R : x > 0}.

Sets and functions 1 Sets The language of sets and functions pervades mathematics, and most of the important operations in mathematics turn out to be functions or to be expressible in terms of functions.

### ON INDUCED SUBGRAPHS WITH ALL DEGREES ODD. 1. Introduction

ON INDUCED SUBGRAPHS WITH ALL DEGREES ODD A.D. SCOTT Abstract. Gallai proved that the vertex set of any graph can be partitioned into two sets, each inducing a subgraph with all degrees even. We prove

### Divisor graphs have arbitrary order and size

Divisor graphs have arbitrary order and size arxiv:math/0606483v1 [math.co] 20 Jun 2006 Le Anh Vinh School of Mathematics University of New South Wales Sydney 2052 Australia Abstract A divisor graph G

### 136 CHAPTER 4. INDUCTION, GRAPHS AND TREES

136 TER 4. INDUCTION, GRHS ND TREES 4.3 Graphs In this chapter we introduce a fundamental structural idea of discrete mathematics, that of a graph. Many situations in the applications of discrete mathematics

### GROUPS SUBGROUPS. Definition 1: An operation on a set G is a function : G G G.

Definition 1: GROUPS An operation on a set G is a function : G G G. Definition 2: A group is a set G which is equipped with an operation and a special element e G, called the identity, such that (i) the

### Chapter 1 - σ-algebras

Page 1 of 17 PROBABILITY 3 - Lecture Notes Chapter 1 - σ-algebras Let Ω be a set of outcomes. We denote by P(Ω) its power set, i.e. the collection of all the subsets of Ω. If the cardinality Ω of Ω is

### Random graphs with a given degree sequence

Sourav Chatterjee (NYU) Persi Diaconis (Stanford) Allan Sly (Microsoft) Let G be an undirected simple graph on n vertices. Let d 1,..., d n be the degrees of the vertices of G arranged in descending order.

### A threshold for the Maker-Breaker clique game

A threshold for the Maker-Breaker clique game Tobias Müller Miloš Stojaković October 7, 01 Abstract We study the Maker-Breaker k-clique game played on the edge set of the random graph G(n, p. In this game,