# Probabilistic Graphical Models

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Probabilistic Graphical Models Raquel Urtasun and Tamir Hazan TTI Chicago April 4, 2011 Raquel Urtasun and Tamir Hazan (TTI-C) Graphical Models April 4, / 22

2 Bayesian Networks and independences Not every distribution independencies can be captured by a directed graph Regularity in the parameterization of the distribution that cannot be captured in the graph structure, e.g., XOR example { 1/12 if x y z = false P(x, y, z) = 1/6 if x y z = true (X Y ) I(P) Z is not independent of X given Y or Y given X. An I-map is the network X Z Y. This is not a perfect map as (X Z) I(P) Symmetric variable-level independencies that are not naturally expressed with a Bayesian network. Independence assumptions imposed by the structure of the DBN are not appropriate, e.g., misconception example Raquel Urtasun and Tamir Hazan (TTI-C) Graphical Models April 4, / 22

3 Misconception example (a) (b) (c) (a) Two independencies: (A C D, B) and (B D A, C) Can we encode this with a BN? (b) First attempt: encodes (A C D, B) but it also implies that (B D A) but dependent given both A, C (c) Second attempt: encodes (A C D, B), but also implies that B and D are marginally independent. Raquel Urtasun and Tamir Hazan (TTI-C) Graphical Models April 4, / 22

4 Undirected graphical models I So far we have seen directed graphical models or Bayesian networks BN do not captured all the independencies, e.g., misconception example, We want a representation that does not require directionality of the influences. We do this via an undirected graph. Undirected graphical models, which are useful in modeling phenomena where the interaction between variables does not have a clear directionality. Often simpler perspective on directed models, in terms of the independence structure and of inference. Raquel Urtasun and Tamir Hazan (TTI-C) Graphical Models April 4, / 22

5 Undirected graphical models II As in BN, the nodes in the graph represent the variables The edges represent direct probabilistic interaction between the neighboring variables How to parametrize the graph? In BN we used CPD (conditional probabilities) to represent distribution of a node given others For undirected graphs, we use a more symmetric parameterization that captures the affinities between related variables. Given a set of random variables X we define a factor as a function from Val(X) to R. The set of variables X is called the scope of the factor. Factors can be negative. In general, we restrict the discussion to positive factors Raquel Urtasun and Tamir Hazan (TTI-C) Graphical Models April 4, / 22

6 Misconception example once more... We can write the joint probability as p(a, B, C, D) = 1 Z φ 1(A, B)φ 2 (B, C)φ 3 (C, D)φ 4 (A, D) Z is the partition function and is used to normalized the probabilities Z = φ 1 (A, B)φ 2 (B, C)φ 3 (C, D)φ 4 (A, D) A,B,C,D It is called function as it depends on the parameters: important for learning. For positive factors, the higher the value of φ, the higher the compatibility. This representation is very flexible. Raquel Urtasun and Tamir Hazan (TTI-C) Graphical Models April 4, / 22

7 Query about probabilities What s the p(b 0 )? Marginalize the other variables! Raquel Urtasun and Tamir Hazan (TTI-C) Graphical Models April 4, / 22

8 Misconception example once more... We can write the joint probability as p(a, B, C, D) = 1 Z φ 1(A, B)φ 2 (B, C)φ 3 (C, D)φ 4 (A, D) Use the joint distribution to query about conditional probabilities by summing out the other variables. Tight connexion between the factorization and the independence properties X Y Z iff p(x, Y, Z) = φ 1 (X, Z)φ 2 (Y, Z) We see that in the example, (A C D, B) and (B D A, C) Raquel Urtasun and Tamir Hazan (TTI-C) Graphical Models April 4, / 22

9 Factors A factor can represent a joint distribution over D by defining φ(d). A factor can represent a CPD p(x D) by defining φ(d X ) But joint and CPD are more restricted, i.e., normalization constraints. Associating parameters over edges is not enough We need to associate factors over sets of nodes, i.e., higher order terms Raquel Urtasun and Tamir Hazan (TTI-C) Graphical Models April 4, / 22

10 Factor product Given 3 disjoint set of variables X, Y, Z, and factors φ 1 (X, Y), φ 2 (Y, Z), the factor product is defined as ψ(x, Y, Z) = φ 1 (X, Y)φ 2 (Y, Z) Raquel Urtasun and Tamir Hazan (TTI-C) Graphical Models April 4, / 22

11 Gibbs distributions and Markov networks A distribution P φ is a Gibbs distribution parameterized with a set of factors φ 1 (D 1 ),, φ m (D m ) if it is defined as P φ (X 1,, X n ) = 1 Z φ 1(D 1 ) φ m (D m ) and the partition function is defined as Z = φ 1 (D 1 ) φ m (D m ) X 1,,,X n The factors do NOT represent marginal probabilities of the variables of their scope. A factor is only one contribution to the joint. A distribution P φ with φ 1 (D 1 ),, φ m (D m ) factorizes over a Markov network H if each D i is a complete subgraph of H The factors that parameterize a Markov network are called clique potentials Raquel Urtasun and Tamir Hazan (TTI-C) Graphical Models April 4, / 22

12 Maximal cliques One can reduce the number of factors by using factors of the maximal cliques This obscures the structure What s the P φ on the left? And on the right? What s the relationship between the factors? Raquel Urtasun and Tamir Hazan (TTI-C) Graphical Models April 4, / 22

13 Example: Pairwise MRF Undirected graphical model very popular in applications such as computer vision: segmentation, stereo, de-noising The graph has only node potentials φ i (X i ) and pairwise potentials φ i,j (X i, X j ) Grids are particularly popular, e.g., pixels in an image with 4-connectivity Raquel Urtasun and Tamir Hazan (TTI-C) Graphical Models April 4, / 22

14 Reduced Markov Networks I Conditioning on an assignment u to a subset of variables U can be done by Eliminating all entries that are inconsistent with the assignment Re-normalizing the remaining entries so that they sum to 1 (Original) (Cond. on c 1 ) Raquel Urtasun and Tamir Hazan (TTI-C) Graphical Models April 4, / 22

15 Reduced Markov Networks Let H be a Markov network over X and let U = u be the context. The reduced network H[u] is a Markov network over the nodes W = X U where we have an edge between X and Y if there is an edge between then in H If U = Grade? If U = {Grade, SAT }? Raquel Urtasun and Tamir Hazan (TTI-C) Graphical Models April 4, / 22

16 Markov Network Independencies I As in BN, the graph encodes a set of independencies. Probabilistic influence flows along the undirected paths in the graph. It is blocked if we condition on the intervening nodes A path X 1 X k is active given the observed variables E X if none of the X i is in E. A set of nodes Z separates X and Y in H, i.e., sep H (X; Y Z), if there exists no active path between any node X X and Y Y given Z. The definition of separation is monotonic if sep H (X; Y Z) then sep H (X; Y Z ) for any Z Z Raquel Urtasun and Tamir Hazan (TTI-C) Graphical Models April 4, / 22

17 Markov Network Independencies II If P is a Gibbs distribution that factorizes over H, then H is an I-map for P, i.e., I (H) I (P) (soundness of separation) Proof: Suppose Z separates X from Y. Then we can write p(x 1,, X n ) = 1 f (X, Z)g(Y, Z) Z A distribution is positive if P(x) > 0 for all x. Hammersley-Clifford theorem: If P is a positive distribution over X and H is an I-map for P, then P is a Gibbs distribution that factorizes over H p(x) = 1 φ c (x c ) Z It is not the case that every pair of nodes that are not separated in H are dependent in every distribution which factorizes over H If X and Y are not separated given Z in H, then X and Y are dependent given Z in some distribution that factorizes over H. Raquel Urtasun and Tamir Hazan (TTI-C) Graphical Models April 4, / 22 c

18 Independence assumptions I In a BN we specify local Markov assumptions and d-separation. In Markov networks we have 1 Global assumption: A set of nodes Z separates X and Y if there is no active path between any node X X and Y Y given Z. I(H) = {(X Y Z) : sep H (X; Y Z)} 2 Pairwise Markov assumption: X and Y are independent give all the other nodes in the graph if no direct connection exists between them I p (H) = {(X Y X {X, Y }) : X Y / X } 3 Markov blanket assumption: X is independent of the rest of the nodes given its neighbors I l (H) = {(X X {X } MB H (X ) MB H (X )) : X X } A set U is a Markov blanket of X if X / U and if U is a minimal set of nodes such that (X X {X } U U) I Raquel Urtasun and Tamir Hazan (TTI-C) Graphical Models April 4, / 22

19 Independence assumptions II (Markov blanket) In general I(H) I l (H) I p (H) If P satisfies I(H), then it satisfies I l (H) If P satisfies I l (H), then it satisfies I p (H) If P is a positive distribution and satisfies I p (H) then it satisfies I l (H) Raquel Urtasun and Tamir Hazan (TTI-C) Graphical Models April 4, / 22

20 From distributions to graphs I The notion of I-map is not enough: as in BN the complete graph is an I-map for any distribution, but does not imply any independencies For a given distribution, we want to construct a minimal I-map based on the local indep. assumptions 1 Pairwise: Add an edge between all pairs of nodes that do NOT satisfy (X Y X {X, Y }) 2 Markov blanket: For each variable X we define the neighbors of X all the nodes that render X independent of the rest of nodes. Define a graph by introducing and edge for all X and all Y MB P (X ). If P is positive distribution, there is a unique Markov blanket of X in I(P), denoted MB P (X ) Raquel Urtasun and Tamir Hazan (TTI-C) Graphical Models April 4, / 22

21 From distributions to graphs II If P is a positive distribution, and let H be the graph defined by introducing an edge {X, Y } for which P does NOT satisfied (X Y X {X, Y }), then H is the unique minimal I-map for P. Minimal I-map is the one that if we remove one edge is not an I-map. Proof: H is an I-map for P since P by construction satisfies I p (P) which for positive distributions equals I(P). To prove that it s minimal, if we eliminate an edge {X, Y } the graph would imply (X Y X {X, Y }), which is false for P, otherwise edge omitted when constructing H. To prove that it s unique: any other I-map must contain the same edges, and it s either equal or contain additional edges, and thus it is not minimal If P is a positive distribution, and for each node let MB P (X ) be a minimal set of nodes U satisfying (X X {X } U U) I. Define H by introducing an edge {X, Y } for all X and all Y MB P (X ). Then H is the unique minimal I-map of P. Raquel Urtasun and Tamir Hazan (TTI-C) Graphical Models April 4, / 22

22 Examples of deterministic relations Not every distribution has a perfect map Even for positive distributions Example is the V-structure, where minimal I-map is the fully connected graph It fails to capture the marginal independence (X Y ) that holds in P Raquel Urtasun and Tamir Hazan (TTI-C) Graphical Models April 4, / 22

### Bayesian vs. Markov Networks

Bayesian vs. Markov Networks Le Song Machine Learning II: dvanced Topics CSE 8803ML, Spring 2012 Conditional Independence ssumptions Local Markov ssumption Global Markov ssumption 𝑋 𝑁𝑜𝑛𝑑𝑒𝑠𝑐𝑒𝑛𝑑𝑎𝑛𝑡𝑋 𝑃𝑎𝑋

### 3. The Junction Tree Algorithms

A Short Course on Graphical Models 3. The Junction Tree Algorithms Mark Paskin mark@paskin.org 1 Review: conditional independence Two random variables X and Y are independent (written X Y ) iff p X ( )

### COS513 LECTURE 5 JUNCTION TREE ALGORITHM

COS513 LECTURE 5 JUNCTION TREE ALGORITHM D. EIS AND V. KOSTINA The junction tree algorithm is the culmination of the way graph theory and probability combine to form graphical models. After we discuss

### The Basics of Graphical Models

The Basics of Graphical Models David M. Blei Columbia University October 3, 2015 Introduction These notes follow Chapter 2 of An Introduction to Probabilistic Graphical Models by Michael Jordan. Many figures

### The Joint Probability Distribution (JPD) of a set of n binary variables involve a huge number of parameters

DEFINING PROILISTI MODELS The Joint Probability Distribution (JPD) of a set of n binary variables involve a huge number of parameters 2 n (larger than 10 25 for only 100 variables). x y z p(x, y, z) 0

### Lecture 11: Graphical Models for Inference

Lecture 11: Graphical Models for Inference So far we have seen two graphical models that are used for inference - the Bayesian network and the Join tree. These two both represent the same joint probability

### Markov random fields and Gibbs measures

Chapter Markov random fields and Gibbs measures 1. Conditional independence Suppose X i is a random element of (X i, B i ), for i = 1, 2, 3, with all X i defined on the same probability space (.F, P).

### 2.3 Scheduling jobs on identical parallel machines

2.3 Scheduling jobs on identical parallel machines There are jobs to be processed, and there are identical machines (running in parallel) to which each job may be assigned Each job = 1,,, must be processed

### CS 188: Artificial Intelligence. Probability recap

CS 188: Artificial Intelligence Bayes Nets Representation and Independence Pieter Abbeel UC Berkeley Many slides over this course adapted from Dan Klein, Stuart Russell, Andrew Moore Conditional probability

### 5 Directed acyclic graphs

5 Directed acyclic graphs (5.1) Introduction In many statistical studies we have prior knowledge about a temporal or causal ordering of the variables. In this chapter we will use directed graphs to incorporate

### Course: Model, Learning, and Inference: Lecture 5

Course: Model, Learning, and Inference: Lecture 5 Alan Yuille Department of Statistics, UCLA Los Angeles, CA 90095 yuille@stat.ucla.edu Abstract Probability distributions on structured representation.

### Bayesian Networks Chapter 14. Mausam (Slides by UW-AI faculty & David Page)

Bayesian Networks Chapter 14 Mausam (Slides by UW-AI faculty & David Page) Bayes Nets In general, joint distribution P over set of variables (X 1 x... x X n ) requires exponential space for representation

### Probabilistic Graphical Models Homework 1: Due January 29, 2014 at 4 pm

Probabilistic Graphical Models 10-708 Homework 1: Due January 29, 2014 at 4 pm Directions. This homework assignment covers the material presented in Lectures 1-3. You must complete all four problems to

### Spatial Statistics Chapter 3 Basics of areal data and areal data modeling

Spatial Statistics Chapter 3 Basics of areal data and areal data modeling Recall areal data also known as lattice data are data Y (s), s D where D is a discrete index set. This usually corresponds to data

### Distributed Structured Prediction for Big Data

Distributed Structured Prediction for Big Data A. G. Schwing ETH Zurich aschwing@inf.ethz.ch T. Hazan TTI Chicago M. Pollefeys ETH Zurich R. Urtasun TTI Chicago Abstract The biggest limitations of learning

### Complexity Theory. IE 661: Scheduling Theory Fall 2003 Satyaki Ghosh Dastidar

Complexity Theory IE 661: Scheduling Theory Fall 2003 Satyaki Ghosh Dastidar Outline Goals Computation of Problems Concepts and Definitions Complexity Classes and Problems Polynomial Time Reductions Examples

### Querying Joint Probability Distributions

Querying Joint Probability Distributions Sargur Srihari srihari@cedar.buffalo.edu 1 Queries of Interest Probabilistic Graphical Models (BNs and MNs) represent joint probability distributions over multiple

### Approximating the Partition Function by Deleting and then Correcting for Model Edges

Approximating the Partition Function by Deleting and then Correcting for Model Edges Arthur Choi and Adnan Darwiche Computer Science Department University of California, Los Angeles Los Angeles, CA 995

### 1. Nondeterministically guess a solution (called a certificate) 2. Check whether the solution solves the problem (called verification)

Some N P problems Computer scientists have studied many N P problems, that is, problems that can be solved nondeterministically in polynomial time. Traditionally complexity question are studied as languages:

### Lecture 4: BK inequality 27th August and 6th September, 2007

CSL866: Percolation and Random Graphs IIT Delhi Amitabha Bagchi Scribe: Arindam Pal Lecture 4: BK inequality 27th August and 6th September, 2007 4. Preliminaries The FKG inequality allows us to lower bound

### Examples and Proofs of Inference in Junction Trees

Examples and Proofs of Inference in Junction Trees Peter Lucas LIAC, Leiden University February 4, 2016 1 Representation and notation Let P(V) be a joint probability distribution, where V stands for a

### Lecture 2: Introduction to belief (Bayesian) networks

Lecture 2: Introduction to belief (Bayesian) networks Conditional independence What is a belief network? Independence maps (I-maps) January 7, 2008 1 COMP-526 Lecture 2 Recall from last time: Conditional

### Life of A Knowledge Base (KB)

Life of A Knowledge Base (KB) A knowledge base system is a special kind of database management system to for knowledge base management. KB extraction: knowledge extraction using statistical models in NLP/ML

### 5.1 Bipartite Matching

CS787: Advanced Algorithms Lecture 5: Applications of Network Flow In the last lecture, we looked at the problem of finding the maximum flow in a graph, and how it can be efficiently solved using the Ford-Fulkerson

### Triangle deletion. Ernie Croot. February 3, 2010

Triangle deletion Ernie Croot February 3, 2010 1 Introduction The purpose of this note is to give an intuitive outline of the triangle deletion theorem of Ruzsa and Szemerédi, which says that if G = (V,

### Part 2: Community Detection

Chapter 8: Graph Data Part 2: Community Detection Based on Leskovec, Rajaraman, Ullman 2014: Mining of Massive Datasets Big Data Management and Analytics Outline Community Detection - Social networks -

### Programming Tools based on Big Data and Conditional Random Fields

Programming Tools based on Big Data and Conditional Random Fields Veselin Raychev Martin Vechev Andreas Krause Department of Computer Science ETH Zurich Zurich Machine Learning and Data Science Meet-up,

### Bayesian Networks. Mausam (Slides by UW-AI faculty)

Bayesian Networks Mausam (Slides by UW-AI faculty) Bayes Nets In general, joint distribution P over set of variables (X 1 x... x X n ) requires exponential space for representation & inference BNs provide

### A Sublinear Bipartiteness Tester for Bounded Degree Graphs

A Sublinear Bipartiteness Tester for Bounded Degree Graphs Oded Goldreich Dana Ron February 5, 1998 Abstract We present a sublinear-time algorithm for testing whether a bounded degree graph is bipartite

### Probability, Conditional Independence

Probability, Conditional Independence June 19, 2012 Probability, Conditional Independence Probability Sample space Ω of events Each event ω Ω has an associated measure Probability of the event P(ω) Axioms

### Logic, Probability and Learning

Logic, Probability and Learning Luc De Raedt luc.deraedt@cs.kuleuven.be Overview Logic Learning Probabilistic Learning Probabilistic Logic Learning Closely following : Russell and Norvig, AI: a modern

### Tutorial on variational approximation methods. Tommi S. Jaakkola MIT AI Lab

Tutorial on variational approximation methods Tommi S. Jaakkola MIT AI Lab tommi@ai.mit.edu Tutorial topics A bit of history Examples of variational methods A brief intro to graphical models Variational

### DATA ANALYSIS II. Matrix Algorithms

DATA ANALYSIS II Matrix Algorithms Similarity Matrix Given a dataset D = {x i }, i=1,..,n consisting of n points in R d, let A denote the n n symmetric similarity matrix between the points, given as where

### p if x = 1 1 p if x = 0

Probability distributions Bernoulli distribution Two possible values (outcomes): 1 (success), 0 (failure). Parameters: p probability of success. Probability mass function: P (x; p) = { p if x = 1 1 p if

### Large-Sample Learning of Bayesian Networks is NP-Hard

Journal of Machine Learning Research 5 (2004) 1287 1330 Submitted 3/04; Published 10/04 Large-Sample Learning of Bayesian Networks is NP-Hard David Maxwell Chickering David Heckerman Christopher Meek Microsoft

### Artificial Intelligence Mar 27, Bayesian Networks 1 P (T D)P (D) + P (T D)P ( D) =

Artificial Intelligence 15-381 Mar 27, 2007 Bayesian Networks 1 Recap of last lecture Probability: precise representation of uncertainty Probability theory: optimal updating of knowledge based on new information

### Minimum Spanning Trees

Minimum Spanning Trees Algorithms and 18.304 Presentation Outline 1 Graph Terminology Minimum Spanning Trees 2 3 Outline Graph Terminology Minimum Spanning Trees 1 Graph Terminology Minimum Spanning Trees

### Structured Learning and Prediction in Computer Vision. Contents

Foundations and Trends R in Computer Graphics and Vision Vol. 6, Nos. 3 4 (2010) 185 365 c 2011 S. Nowozin and C. H. Lampert DOI: 10.1561/0600000033 Structured Learning and Prediction in Computer Vision

### Approximation Algorithms: LP Relaxation, Rounding, and Randomized Rounding Techniques. My T. Thai

Approximation Algorithms: LP Relaxation, Rounding, and Randomized Rounding Techniques My T. Thai 1 Overview An overview of LP relaxation and rounding method is as follows: 1. Formulate an optimization

### Generating models of a matched formula with a polynomial delay

Generating models of a matched formula with a polynomial delay Petr Savicky Institute of Computer Science, Academy of Sciences of Czech Republic, Pod Vodárenskou Věží 2, 182 07 Praha 8, Czech Republic

### Handout #Ch7 San Skulrattanakulchai Gustavus Adolphus College Dec 6, 2010. Chapter 7: Digraphs

MCS-236: Graph Theory Handout #Ch7 San Skulrattanakulchai Gustavus Adolphus College Dec 6, 2010 Chapter 7: Digraphs Strong Digraphs Definitions. A digraph is an ordered pair (V, E), where V is the set

### Bayesian Networks. Read R&N Ch. 14.1-14.2. Next lecture: Read R&N 18.1-18.4

Bayesian Networks Read R&N Ch. 14.1-14.2 Next lecture: Read R&N 18.1-18.4 You will be expected to know Basic concepts and vocabulary of Bayesian networks. Nodes represent random variables. Directed arcs

### R u t c o r Research R e p o r t. Boolean functions with a simple certificate for CNF complexity. Ondřej Čepeka Petr Kučera b Petr Savický c

R u t c o r Research R e p o r t Boolean functions with a simple certificate for CNF complexity Ondřej Čepeka Petr Kučera b Petr Savický c RRR 2-2010, January 24, 2010 RUTCOR Rutgers Center for Operations

### Network (Tree) Topology Inference Based on Prüfer Sequence

Network (Tree) Topology Inference Based on Prüfer Sequence C. Vanniarajan and Kamala Krithivasan Department of Computer Science and Engineering Indian Institute of Technology Madras Chennai 600036 vanniarajanc@hcl.in,

### Clique coloring B 1 -EPG graphs

Clique coloring B 1 -EPG graphs Flavia Bonomo a,c, María Pía Mazzoleni b,c, and Maya Stein d a Departamento de Computación, FCEN-UBA, Buenos Aires, Argentina. b Departamento de Matemática, FCE-UNLP, La

### Foundations of Computing Discrete Mathematics Solutions to exercises for week 2

Foundations of Computing Discrete Mathematics Solutions to exercises for week 2 Agata Murawska (agmu@itu.dk) September 16, 2013 Note. The solutions presented here are usually one of many possiblities.

### Message-passing sequential detection of multiple change points in networks

Message-passing sequential detection of multiple change points in networks Long Nguyen, Arash Amini Ram Rajagopal University of Michigan Stanford University ISIT, Boston, July 2012 Nguyen/Amini/Rajagopal

### Graphical Models as Block-Tree Graphs

Graphical Models as Block-Tree Graphs 1 Divyanshu Vats and José M. F. Moura arxiv:1007.0563v2 [stat.ml] 13 Nov 2010 Abstract We introduce block-tree graphs as a framework for deriving efficient algorithms

### Lecture 7: Approximation via Randomized Rounding

Lecture 7: Approximation via Randomized Rounding Often LPs return a fractional solution where the solution x, which is supposed to be in {0, } n, is in [0, ] n instead. There is a generic way of obtaining

### CS 598CSC: Combinatorial Optimization Lecture date: 2/4/2010

CS 598CSC: Combinatorial Optimization Lecture date: /4/010 Instructor: Chandra Chekuri Scribe: David Morrison Gomory-Hu Trees (The work in this section closely follows [3]) Let G = (V, E) be an undirected

### Information Granulation and Approximation in a Decision-theoretic Model of Rough Sets

Information Granulation and Approximation in a Decision-theoretic Model of Rough Sets Y.Y. Yao Department of Computer Science University of Regina Regina, Saskatchewan Canada S4S 0A2 E-mail: yyao@cs.uregina.ca

### ABC methods for model choice in Gibbs random fields

ABC methods for model choice in Gibbs random fields Jean-Michel Marin Institut de Mathématiques et Modélisation Université Montpellier 2 Joint work with Aude Grelaud, Christian Robert, François Rodolphe,

### Fairness in Routing and Load Balancing

Fairness in Routing and Load Balancing Jon Kleinberg Yuval Rabani Éva Tardos Abstract We consider the issue of network routing subject to explicit fairness conditions. The optimization of fairness criteria

### Online Model Predictive Control of a Robotic System by Combining Simulation and Optimization

Mohammad Rokonuzzaman Pappu Online Model Predictive Control of a Robotic System by Combining Simulation and Optimization School of Electrical Engineering Department of Electrical Engineering and Automation

### Definition 11.1. Given a graph G on n vertices, we define the following quantities:

Lecture 11 The Lovász ϑ Function 11.1 Perfect graphs We begin with some background on perfect graphs. graphs. First, we define some quantities on Definition 11.1. Given a graph G on n vertices, we define

### Tools for parsimonious edge-colouring of graphs with maximum degree three. J.L. Fouquet and J.M. Vanherpe. Rapport n o RR-2010-10

Tools for parsimonious edge-colouring of graphs with maximum degree three J.L. Fouquet and J.M. Vanherpe LIFO, Université d Orléans Rapport n o RR-2010-10 Tools for parsimonious edge-colouring of graphs

### Resource Allocation with Time Intervals

Resource Allocation with Time Intervals Andreas Darmann Ulrich Pferschy Joachim Schauer Abstract We study a resource allocation problem where jobs have the following characteristics: Each job consumes

### Lecture 15 An Arithmetic Circuit Lowerbound and Flows in Graphs

CSE599s: Extremal Combinatorics November 21, 2011 Lecture 15 An Arithmetic Circuit Lowerbound and Flows in Graphs Lecturer: Anup Rao 1 An Arithmetic Circuit Lower Bound An arithmetic circuit is just like

### CSC 373: Algorithm Design and Analysis Lecture 16

CSC 373: Algorithm Design and Analysis Lecture 16 Allan Borodin February 25, 2013 Some materials are from Stephen Cook s IIT talk and Keven Wayne s slides. 1 / 17 Announcements and Outline Announcements

### Graph definition Degree, in, out degree, oriented graph. Complete, regular, bipartite graph. Graph representation, connectivity, adjacency.

Mária Markošová Graph definition Degree, in, out degree, oriented graph. Complete, regular, bipartite graph. Graph representation, connectivity, adjacency. Isomorphism of graphs. Paths, cycles, trials.

### Cycles and clique-minors in expanders

Cycles and clique-minors in expanders Benny Sudakov UCLA and Princeton University Expanders Definition: The vertex boundary of a subset X of a graph G: X = { all vertices in G\X with at least one neighbor

### Exponential time algorithms for graph coloring

Exponential time algorithms for graph coloring Uriel Feige Lecture notes, March 14, 2011 1 Introduction Let [n] denote the set {1,..., k}. A k-labeling of vertices of a graph G(V, E) is a function V [k].

### Informatics 2D Reasoning and Agents Semester 2, 2015-16

Informatics 2D Reasoning and Agents Semester 2, 2015-16 Alex Lascarides alex@inf.ed.ac.uk Lecture 29 Decision Making Under Uncertainty 24th March 2016 Informatics UoE Informatics 2D 1 Where are we? Last

### Private Approximation of Clustering and Vertex Cover

Private Approximation of Clustering and Vertex Cover Amos Beimel, Renen Hallak, and Kobbi Nissim Department of Computer Science, Ben-Gurion University of the Negev Abstract. Private approximation of search

### Determination of the normalization level of database schemas through equivalence classes of attributes

Computer Science Journal of Moldova, vol.17, no.2(50), 2009 Determination of the normalization level of database schemas through equivalence classes of attributes Cotelea Vitalie Abstract In this paper,

### Maximum Hitting Time for Random Walks on Graphs. Graham Brightwell, Cambridge University Peter Winkler*, Emory University

Maximum Hitting Time for Random Walks on Graphs Graham Brightwell, Cambridge University Peter Winkler*, Emory University Abstract. For x and y vertices of a connected graph G, let T G (x, y denote the

### Studying Relationships

Binary Relations Problem Problem Set Set Two Two checkpoint checkpoint due due in in the the box box up up front front if if you're you're using using a a late late period. period. Studying Relationships

### Approximation Algorithms

Approximation Algorithms or: How I Learned to Stop Worrying and Deal with NP-Completeness Ong Jit Sheng, Jonathan (A0073924B) March, 2012 Overview Key Results (I) General techniques: Greedy algorithms

### 6.852: Distributed Algorithms Fall, 2009. Class 2

.8: Distributed Algorithms Fall, 009 Class Today s plan Leader election in a synchronous ring: Lower bound for comparison-based algorithms. Basic computation in general synchronous networks: Leader election

### L(n) OA(n, 3): if L is a latin square with colour set [n] then the following is an O(n, 3):

9 Latin Squares Latin Square: A Latin Square of order n is an n n array where each cell is occupied by one of n colours with the property that each colour appears exactly once in each row and in each column.

### SEQUENCES OF MAXIMAL DEGREE VERTICES IN GRAPHS. Nickolay Khadzhiivanov, Nedyalko Nenov

Serdica Math. J. 30 (2004), 95 102 SEQUENCES OF MAXIMAL DEGREE VERTICES IN GRAPHS Nickolay Khadzhiivanov, Nedyalko Nenov Communicated by V. Drensky Abstract. Let Γ(M) where M V (G) be the set of all vertices

### 13.3 Inference Using Full Joint Distribution

191 The probability distribution on a single variable must sum to 1 It is also true that any joint probability distribution on any set of variables must sum to 1 Recall that any proposition a is equivalent

### Advanced Spatial Statistics Fall 2012 NCSU. Fuentes Lecture notes

Advanced Spatial Statistics Fall 2012 NCSU Fuentes Lecture notes Areal unit data 2 Areal Modelling Areal unit data Key Issues Is there spatial pattern? Spatial pattern implies that observations from units

### Proximal mapping via network optimization

L. Vandenberghe EE236C (Spring 23-4) Proximal mapping via network optimization minimum cut and maximum flow problems parametric minimum cut problem application to proximal mapping Introduction this lecture:

### Satisfiability Checking

Satisfiability Checking SAT-Solving Prof. Dr. Erika Ábrahám Theory of Hybrid Systems Informatik 2 WS 10/11 Prof. Dr. Erika Ábrahám - Satisfiability Checking 1 / 40 A basic SAT algorithm Assume the CNF

### Finding the M Most Probable Configurations Using Loopy Belief Propagation

Finding the M Most Probable Configurations Using Loopy Belief Propagation Chen Yanover and Yair Weiss School of Computer Science and Engineering The Hebrew University of Jerusalem 91904 Jerusalem, Israel

### Notes on Matrix Multiplication and the Transitive Closure

ICS 6D Due: Wednesday, February 25, 2015 Instructor: Sandy Irani Notes on Matrix Multiplication and the Transitive Closure An n m matrix over a set S is an array of elements from S with n rows and m columns.

### Graph Mining and Social Network Analysis

Graph Mining and Social Network Analysis Data Mining and Text Mining (UIC 583 @ Politecnico di Milano) References Jiawei Han and Micheline Kamber, "Data Mining: Concepts and Techniques", The Morgan Kaufmann

### Homework MA 725 Spring, 2012 C. Huneke SELECTED ANSWERS

Homework MA 725 Spring, 2012 C. Huneke SELECTED ANSWERS 1.1.25 Prove that the Petersen graph has no cycle of length 7. Solution: There are 10 vertices in the Petersen graph G. Assume there is a cycle C

### Introduction to Algorithms. Part 3: P, NP Hard Problems

Introduction to Algorithms Part 3: P, NP Hard Problems 1) Polynomial Time: P and NP 2) NP-Completeness 3) Dealing with Hard Problems 4) Lower Bounds 5) Books c Wayne Goddard, Clemson University, 2004 Chapter

### GENERATING LOW-DEGREE 2-SPANNERS

SIAM J. COMPUT. c 1998 Society for Industrial and Applied Mathematics Vol. 27, No. 5, pp. 1438 1456, October 1998 013 GENERATING LOW-DEGREE 2-SPANNERS GUY KORTSARZ AND DAVID PELEG Abstract. A k-spanner

### Planar Graph and Trees

Dr. Nahid Sultana December 16, 2012 Tree Spanning Trees Minimum Spanning Trees Maps and Regions Eulers Formula Nonplanar graph Dual Maps and the Four Color Theorem Tree Spanning Trees Minimum Spanning

### (67902) Topics in Theory and Complexity Nov 2, 2006. Lecture 7

(67902) Topics in Theory and Complexity Nov 2, 2006 Lecturer: Irit Dinur Lecture 7 Scribe: Rani Lekach 1 Lecture overview This Lecture consists of two parts In the first part we will refresh the definition

### MIX-DECOMPOSITON OF THE COMPLETE GRAPH INTO DIRECTED FACTORS OF DIAMETER 2 AND UNDIRECTED FACTORS OF DIAMETER 3. University of Split, Croatia

GLASNIK MATEMATIČKI Vol. 38(58)(2003), 2 232 MIX-DECOMPOSITON OF THE COMPLETE GRAPH INTO DIRECTED FACTORS OF DIAMETER 2 AND UNDIRECTED FACTORS OF DIAMETER 3 Damir Vukičević University of Split, Croatia

### Best Monotone Degree Bounds for Various Graph Parameters

Best Monotone Degree Bounds for Various Graph Parameters D. Bauer Department of Mathematical Sciences Stevens Institute of Technology Hoboken, NJ 07030 S. L. Hakimi Department of Electrical and Computer

### Finding and counting given length cycles

Finding and counting given length cycles Noga Alon Raphael Yuster Uri Zwick Abstract We present an assortment of methods for finding and counting simple cycles of a given length in directed and undirected

### CMSC 858T: Randomized Algorithms Spring 2003 Handout 8: The Local Lemma

CMSC 858T: Randomized Algorithms Spring 2003 Handout 8: The Local Lemma Please Note: The references at the end are given for extra reading if you are interested in exploring these ideas further. You are

### 1. Relevant standard graph theory

Color identical pairs in 4-chromatic graphs Asbjørn Brændeland I argue that, given a 4-chromatic graph G and a pair of vertices {u, v} in G, if the color of u equals the color of v in every 4-coloring

### On an anti-ramsey type result

On an anti-ramsey type result Noga Alon, Hanno Lefmann and Vojtĕch Rödl Abstract We consider anti-ramsey type results. For a given coloring of the k-element subsets of an n-element set X, where two k-element

### / Approximation Algorithms Lecturer: Michael Dinitz Topic: Steiner Tree and TSP Date: 01/29/15 Scribe: Katie Henry

600.469 / 600.669 Approximation Algorithms Lecturer: Michael Dinitz Topic: Steiner Tree and TSP Date: 01/29/15 Scribe: Katie Henry 2.1 Steiner Tree Definition 2.1.1 In the Steiner Tree problem the input

### Applied Algorithm Design Lecture 5

Applied Algorithm Design Lecture 5 Pietro Michiardi Eurecom Pietro Michiardi (Eurecom) Applied Algorithm Design Lecture 5 1 / 86 Approximation Algorithms Pietro Michiardi (Eurecom) Applied Algorithm Design

### Variable Elimination 1

Variable Elimination 1 Inference Exact inference Enumeration Variable elimination Junction trees and belief propagation Approximate inference Loopy belief propagation Sampling methods: likelihood weighting,

### Why? A central concept in Computer Science. Algorithms are ubiquitous.

Analysis of Algorithms: A Brief Introduction Why? A central concept in Computer Science. Algorithms are ubiquitous. Using the Internet (sending email, transferring files, use of search engines, online

### Discrete Mathematics & Mathematical Reasoning Chapter 10: Graphs

Discrete Mathematics & Mathematical Reasoning Chapter 10: Graphs Kousha Etessami U. of Edinburgh, UK Kousha Etessami (U. of Edinburgh, UK) Discrete Mathematics (Chapter 6) 1 / 13 Overview Graphs and Graph

### 1 Basic Definitions and Concepts in Graph Theory

CME 305: Discrete Mathematics and Algorithms 1 Basic Definitions and Concepts in Graph Theory A graph G(V, E) is a set V of vertices and a set E of edges. In an undirected graph, an edge is an unordered

### On the independence number of graphs with maximum degree 3

On the independence number of graphs with maximum degree 3 Iyad A. Kanj Fenghui Zhang Abstract Let G be an undirected graph with maximum degree at most 3 such that G does not contain any of the three graphs

### Lesson 3. Algebraic graph theory. Sergio Barbarossa. Rome - February 2010

Lesson 3 Algebraic graph theory Sergio Barbarossa Basic notions Definition: A directed graph (or digraph) composed by a set of vertices and a set of edges We adopt the convention that the information flows

### On the Multiple Unicast Network Coding Conjecture

On the Multiple Unicast Networ Coding Conjecture Michael Langberg Computer Science Division Open University of Israel Raanana 43107, Israel miel@openu.ac.il Muriel Médard Research Laboratory of Electronics