# COS513 LECTURE 5 JUNCTION TREE ALGORITHM

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 COS513 LECTURE 5 JUNCTION TREE ALGORITHM D. EIS AND V. KOSTINA The junction tree algorithm is the culmination of the way graph theory and probability combine to form graphical models. After we discuss the junction tree algorithm, we will move on to developing the models for data analysis. 1. ALGORITHM OUTLINE The junction tree algorithm has many different moving parts and we will describe each one separately and all the pieces will come together by the end. The goal of the algorithm is to parameterize the model such that it is easy to calculate marginals. We start with a graphical model and then do the following steps: (1) Moralize the graph (2) Triangulate the graph (3) Build a junction tree (4) Apply the message passing algorithm When this algorithm is done we are left with the quantities needed to compute the desired marginals. The idea behind the algorithm is to use independence properties of the graph to decompose general probability calculations into a linked set of local computations, along the lines of what we did in the elimination and message passing algorithms, but here we are switching from an algebraic process of inference to an inference data structure. This algorithm will work with any model that is described by a graph. Now we return to the graph we have looked at since the beginning (Fig. 1). Remember, when we do elimination on this graph, we first moralize the graph by connecting all unconnected parents. After this we make the graph an undirected graph. After applying eliminate, we can construct the reconstituted graph (Fig. 2), which is triangulated, i.e. for any given cycle there is an edge between any two non-successive nodes in the cycle. Previously we used the elimination algorithm to obtain marginals. We also employed the tree propagation algorithm to reuse intermediate factors 1

2 2 D. EIS AND V. KOSTINA X 2 X 4 X 1 X 6 X 3 X 5 FIGURE 1. The six-node example from previous lectures. X 2 X 4 X 1 X 6 X 3 X 5 FIGURE 2. Reconstituted graph. in a tree graph. The junction tree allows us to reuse intermediate factors in any graphical model. The first step in moving beyond the elimination algorithm is to allocate storage for the intermediate factors. Toward this end, we form a clique tree out of the elimination cliques in the graph. Each intermediate factor is associated with one of the cliques, so the clique tree represents the storage we need to do elimination. Moreover, it illustrates the information flow of the elimination procedure - we can follow what happens in elimination by examining the clique tree. An example of a clique tree for our graph using the elimination ordering {6,5,4,3,2,1} is shown in Fig. 3. At this point we note that the clique tree depends on the chosen ordering. We introduce the notion of a separator set, which is just the intersection of two adjacent cliques. We show these sets in boxes separating two cliques explicitly in Fig. 3. Note that members of separation sets correspond to the parameters of the intermediate factors. For example, separation set {X 1 } corresponds to the parameter X 1 of the intermediate factor which arises

3 COS513 LECTURE 5 JUNCTION TREE ALGORITHM 3 X 2 X 2 X 4 X 1 X 1 X 1 X 2 X 1 X 2 X 1 X 2 X 3 X 2 X 3 X 2 X 3 X 5 X 2 X 5 X 2 X 5 X 6 FIGURE 3. A clique tree with separator sets using elimination ordering {6,5,4,3,2,1}. after removing {X 2 }. Later we will define potentials on these separator sets. As a preview, we note that this tree in Fig. 3 has the junction tree property: if a variable is in two cliques then it is in every clique along the path connecting the two cliques. 2. CLIQUE POTENTIALS Let G = {V, E} be a graphical model and let C be a set of cliques, where a clique is a completely connected subset of the graph. Then for each c C, define a clique potential ψ c (X c ). Define the joint probability as in the undirected model: (1) p(x) = 1 ψ c (X c ) Z c C Define C to be the set of maximal cliques, meaning that no member of C is a subset of another member of C. Eventually, we will adjust the clique potentials, ψ c s, to be marginals, so that we can answer general inferential questions about our model. 3. INITIALIZING CLIQUE POTENTIALS The first step we do is to initialize the clique potentials. We note now that the clique potential assigned to each clique is not necessarily the same as the clique potential in the undirected graph. In the case of the undirected graph, if the potentials that are defined on the undirected graph are all on maximal cliques then the clique potentials are just the potentials defined on the maximal cliques. If the potentials are not on maximal cliques, then we let the clique potential equal the product of potentials on subcliques, with the caveat that each potential can only be used once in these definitions, so there will be no repetitions. This ensures that the joint probability, the product of the clique potentials, is the same as the product of the potentials. For example, in Fig. 4, if the potentials are defined on the

4 4 D. EIS AND V. KOSTINA pairwise cliques, then we can define the clique potentials as follows: (2) (3) ψ ABC (A, B, C) = ψ AB (A, B)ψ BC (B, C)ψ AC (A, C) ψ BCD (B, C, D) = ψ CD (C, D)ψ BD (B, D) You can confirm that you must use each potential exactly once, otherwise the joint will not be the same. A B C D FIGURE 4. A four-node model which we assume is parameterized with pairwise potentials ψ AB, ψ AC, ψ BC, ψ BD and ψ CD. To initialize the clique potentials for directed graphs, first moralize the graph. If we do this to the graph in Fig. 1 then we get the graph shown in Fig. 2. The maximal cliques are: {x 2, x 5, x 6 }, {x 2, x 4 }, {x 1, x 3 }, {x 1, x 2 }, {x 3, x 5 }. Then we define the clique potentials to be the product of conditional probabilities such that each conditional probability only involves nodes that are in the maximal clique this potential is being defined on and such that each conditional probability is assigned to exactly one clique potential. We can define the clique potentials as follows for the graph in Fig. 1: (4) ψ 24 (x 2, x 4 ) = p(x 4 x 2 ) ψ 12 (x 1, x 2 ) = p(x 1 )p(x 2 x 1 ) ψ 13 (x 1, x 3 ) = p(x 3 x 1 ) ψ 35 (x 3, x 5 ) = p(x 5 x 3 ) ψ 256 (x 2, x 5, x 6 ) = p(x 6 x 2, x 5 ) To summarize, we initialize clique potentials with original undirected potentials or with conditional probabilities tables from the moralized graph. 4. EVIDENCE Suppose nodes are divided into subsets H and E, where E contains evidence nodes and H contains everything else. How do we compute p(x H x E )? As it turns out, this problem is no different in principle from the calculation

5 COS513 LECTURE 5 JUNCTION TREE ALGORITHM 5 of marginal probabilities from (1). The following trick will do the job: we add δ(x E, x E ) to one clique potential containing E, where x E is observed value. Then computing the right hand side of (1) we will get p(x H, x E ). The only nuance is that the normalization constant Z remains without δ s since it is a property of the distribution and does not depend on evidence. Let us now focus on computing clique potentials. 5. COMPUTING CLIQUE POTENTIALS Our goal is to compute the marginal p(x F ) for F C, C C. The junction tree algorithm uses information flow in the graph to adjust the clique potentials to give this marginal probability. When the algorithm terminates, we ll have ψ C (x C ) = p(x C ) (or p(x C, x E ) if we have evidence). Let us consider the Markov chain example in Fig. 5. A B C (5) FIGURE 5. A three-node model we use to illustrate local consistency. The clique potentials for this model are given by: ψ AB (A, B) = p(a)p(b A) = p(a, B) ψ BC (B, C) = p(c B) Clique potential ψ AB already corresponds to the marginal probability p(a, B). We want to make ψ BC a marginal as well. Naively, we sum out A from ψ AB : (6) ψ AB = p(b) Further, multiplying ψ BC by this p(b) gives the new ψ BC : (7) A ψ BC = p(c B)p(B) = p(b, C) Now, both clique potentials are marginals. But here is the problem: we messed up the joint (1)! The junction tree algorithm will solve this problem. It manipulates the clique potentials to be marginals, but at the same time it does not alter the joint distribution. Moreover, it maintains both local and global consistency. Local consistency means that the neighboring clique potentials agree when the marginal of their common node is calculated from either of them. For example, in Fig. 5 suppose we adjusted the clique potentials ψ AB and ψ BC to be p(a, B) and p(b, C). Local consistency means that when we

6 6 D. EIS AND V. KOSTINA sum out A from ψ AB it gives the same p(b) as when we sum out C from ψ BC. Global consistency means that any two cliques containing the same node will agree on the marginal of that node. 6. SEPARATOR SETS To maintain the joint p(x) like the junction tree algorithm does, we extend our representation to include separator sets, and we define potentials on these separator sets: (8) p(x) = c C ψ c(x c ) s S ψ s(x s ) Here S includes all separator sets and ψ s (X s ) are separator potentials. The idea is to adjust the clique potentials to obtains marginals, and to adjust the ψ s s to maintain the joint, p(x). For example, extended representation for the Markov chain in Fig. 5 is: p(a, B, C) = p(a)p(b A)p(C B) = p(a, B)p(C B) = p(a, B)p(C, B), p(b) where the numerator, p(b), is a separator potential. We can find this extended representation for any probability distribution. (1) It includes all distributions in Eq. (1). (2) It contains no others distributions.

### 3. The Junction Tree Algorithms

A Short Course on Graphical Models 3. The Junction Tree Algorithms Mark Paskin mark@paskin.org 1 Review: conditional independence Two random variables X and Y are independent (written X Y ) iff p X ( )

### Lecture 10: Probability Propagation in Join Trees

Lecture 10: Probability Propagation in Join Trees In the last lecture we looked at how the variables of a Bayesian network could be grouped together in cliques of dependency, so that probability propagation

### 3. Markov property. Markov property for MRFs. Hammersley-Clifford theorem. Markov property for Bayesian networks. I-map, P-map, and chordal graphs

Markov property 3-3. Markov property Markov property for MRFs Hammersley-Clifford theorem Markov property for ayesian networks I-map, P-map, and chordal graphs Markov Chain X Y Z X = Z Y µ(x, Y, Z) = f(x,

### Machine Learning

Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University February 23, 2015 Today: Graphical models Bayes Nets: Representing distributions Conditional independencies

### Probabilistic Graphical Models

Probabilistic Graphical Models Raquel Urtasun and Tamir Hazan TTI Chicago April 4, 2011 Raquel Urtasun and Tamir Hazan (TTI-C) Graphical Models April 4, 2011 1 / 22 Bayesian Networks and independences

### The Basics of Graphical Models

The Basics of Graphical Models David M. Blei Columbia University October 3, 2015 Introduction These notes follow Chapter 2 of An Introduction to Probabilistic Graphical Models by Michael Jordan. Many figures

### Bayesian vs. Markov Networks

Bayesian vs. Markov Networks Le Song Machine Learning II: dvanced Topics CSE 8803ML, Spring 2012 Conditional Independence ssumptions Local Markov ssumption Global Markov ssumption 𝑋 𝑁𝑜𝑛𝑑𝑒𝑠𝑐𝑒𝑛𝑑𝑎𝑛𝑡𝑋 𝑃𝑎𝑋

### Artificial Intelligence Mar 27, Bayesian Networks 1 P (T D)P (D) + P (T D)P ( D) =

Artificial Intelligence 15-381 Mar 27, 2007 Bayesian Networks 1 Recap of last lecture Probability: precise representation of uncertainty Probability theory: optimal updating of knowledge based on new information

### Examples and Proofs of Inference in Junction Trees

Examples and Proofs of Inference in Junction Trees Peter Lucas LIAC, Leiden University February 4, 2016 1 Representation and notation Let P(V) be a joint probability distribution, where V stands for a

### Lecture 11: Graphical Models for Inference

Lecture 11: Graphical Models for Inference So far we have seen two graphical models that are used for inference - the Bayesian network and the Join tree. These two both represent the same joint probability

### Introduction to Discrete Probability Theory and Bayesian Networks

Introduction to Discrete Probability Theory and Bayesian Networks Dr Michael Ashcroft September 15, 2011 This document remains the property of Inatas. Reproduction in whole or in part without the written

### Lecture Notes on Spanning Trees

Lecture Notes on Spanning Trees 15-122: Principles of Imperative Computation Frank Pfenning Lecture 26 April 26, 2011 1 Introduction In this lecture we introduce graphs. Graphs provide a uniform model

### CSPs: Arc Consistency

CSPs: Arc Consistency CPSC 322 CSPs 3 Textbook 4.5 CSPs: Arc Consistency CPSC 322 CSPs 3, Slide 1 Lecture Overview 1 Recap 2 Consistency 3 Arc Consistency CSPs: Arc Consistency CPSC 322 CSPs 3, Slide 2

### Variable Elimination 1

Variable Elimination 1 Inference Exact inference Enumeration Variable elimination Junction trees and belief propagation Approximate inference Loopy belief propagation Sampling methods: likelihood weighting,

### Graphical Models as Block-Tree Graphs

Graphical Models as Block-Tree Graphs 1 Divyanshu Vats and José M. F. Moura arxiv:1007.0563v2 [stat.ml] 13 Nov 2010 Abstract We introduce block-tree graphs as a framework for deriving efficient algorithms

### Solutions to Exercises 8

Discrete Mathematics Lent 2009 MA210 Solutions to Exercises 8 (1) Suppose that G is a graph in which every vertex has degree at least k, where k 1, and in which every cycle contains at least 4 vertices.

### Part 2: Community Detection

Chapter 8: Graph Data Part 2: Community Detection Based on Leskovec, Rajaraman, Ullman 2014: Mining of Massive Datasets Big Data Management and Analytics Outline Community Detection - Social networks -

### The Joint Probability Distribution (JPD) of a set of n binary variables involve a huge number of parameters

DEFINING PROILISTI MODELS The Joint Probability Distribution (JPD) of a set of n binary variables involve a huge number of parameters 2 n (larger than 10 25 for only 100 variables). x y z p(x, y, z) 0

### Course: Model, Learning, and Inference: Lecture 5

Course: Model, Learning, and Inference: Lecture 5 Alan Yuille Department of Statistics, UCLA Los Angeles, CA 90095 yuille@stat.ucla.edu Abstract Probability distributions on structured representation.

### STAT 270 Probability Basics

STAT 270 Probability Basics Richard Lockhart Simon Fraser University Spring 2015 Surrey 1/28 Purposes of These Notes Jargon: experiment, sample space, outcome, event. Set theory ideas and notation: intersection,

### Graphs and Network Flows IE411 Lecture 1

Graphs and Network Flows IE411 Lecture 1 Dr. Ted Ralphs IE411 Lecture 1 1 References for Today s Lecture Required reading Sections 17.1, 19.1 References AMO Chapter 1 and Section 2.1 and 2.2 IE411 Lecture

### Probabilistic Graphical Models Final Exam

Introduction to Probabilistic Graphical Models Final Exam, March 2007 1 Probabilistic Graphical Models Final Exam Please fill your name and I.D.: Name:... I.D.:... Duration: 3 hours. Guidelines: 1. The

### Markov chains and Markov Random Fields (MRFs)

Markov chains and Markov Random Fields (MRFs) 1 Why Markov Models We discuss Markov models now. This is the simplest statistical model in which we don t assume that all variables are independent; we assume

### 1. Relevant standard graph theory

Color identical pairs in 4-chromatic graphs Asbjørn Brændeland I argue that, given a 4-chromatic graph G and a pair of vertices {u, v} in G, if the color of u equals the color of v in every 4-coloring

### Machine Learning

Machine Learning 10-701 Tom M. Mitchell Machine Learning Department Carnegie Mellon University February 8, 2011 Today: Graphical models Bayes Nets: Representing distributions Conditional independencies

### 2) Write in detail the issues in the design of code generator.

COMPUTER SCIENCE AND ENGINEERING VI SEM CSE Principles of Compiler Design Unit-IV Question and answers UNIT IV CODE GENERATION 9 Issues in the design of code generator The target machine Runtime Storage

### A rational number is a number that can be written as where a and b are integers and b 0.

S E L S O N Rational Numbers Goal: Perform operations on rational numbers. Vocabulary Rational number: Additive inverse: A rational number is a number that can be a written as where a and b are integers

### SYSM 6304: Risk and Decision Analysis Lecture 5: Methods of Risk Analysis

SYSM 6304: Risk and Decision Analysis Lecture 5: Methods of Risk Analysis M. Vidyasagar Cecil & Ida Green Chair The University of Texas at Dallas Email: M.Vidyasagar@utdallas.edu October 17, 2015 Outline

### Lecture 16 : Relations and Functions DRAFT

CS/Math 240: Introduction to Discrete Mathematics 3/29/2011 Lecture 16 : Relations and Functions Instructor: Dieter van Melkebeek Scribe: Dalibor Zelený DRAFT In Lecture 3, we described a correspondence

### Click on the links below to jump directly to the relevant section

Click on the links below to jump directly to the relevant section What is algebra? Operations with algebraic terms Mathematical properties of real numbers Order of operations What is Algebra? Algebra is

### PRIMARY CONTENT MODULE Algebra I -Linear Equations & Inequalities T-71. Applications. F = mc + b.

PRIMARY CONTENT MODULE Algebra I -Linear Equations & Inequalities T-71 Applications The formula y = mx + b sometimes appears with different symbols. For example, instead of x, we could use the letter C.

### Lecture 7: NP-Complete Problems

IAS/PCMI Summer Session 2000 Clay Mathematics Undergraduate Program Basic Course on Computational Complexity Lecture 7: NP-Complete Problems David Mix Barrington and Alexis Maciel July 25, 2000 1. Circuit

### Graph Theory. Directed and undirected graphs make useful mental models for many situations. These objects are loosely defined as follows:

Graph Theory Directed and undirected graphs make useful mental models for many situations. These objects are loosely defined as follows: Definition An undirected graph is a (finite) set of nodes, some

### COMPRESSION SPRINGS: STANDARD SERIES (INCH)

: STANDARD SERIES (INCH) LC 014A 01 0.250 6.35 11.25 0.200 0.088 2.24 F F M LC 014A 02 0.313 7.94 8.90 0.159 0.105 2.67 F F M LC 014A 03 0.375 9.52 7.10 0.126 0.122 3.10 F F M LC 014A 04 0.438 11.11 6.00

### Computer Organization I. Lecture 8: Boolean Algebra and Circuit Optimization

Computer Organization I Lecture 8: Boolean Algebra and Circuit Optimization Overview The simplification from SOM to SOP and their circuit implementation Basics of Logic Circuit Optimization: Cost Criteria

### Minimum Spanning Trees

Minimum Spanning Trees Algorithms and 18.304 Presentation Outline 1 Graph Terminology Minimum Spanning Trees 2 3 Outline Graph Terminology Minimum Spanning Trees 1 Graph Terminology Minimum Spanning Trees

### State Spaces Graph Search Searching. Graph Search. CPSC 322 Search 2. Textbook 3.4. Graph Search CPSC 322 Search 2, Slide 1

Graph Search CPSC 322 Search 2 Textbook 3.4 Graph Search CPSC 322 Search 2, Slide 1 Lecture Overview 1 State Spaces 2 Graph Search 3 Searching Graph Search CPSC 322 Search 2, Slide 2 State Spaces Idea:

### BIN504 - Lecture XI. Bayesian Inference & Markov Chains

BIN504 - Lecture XI Bayesian Inference & References: Lee, Chp. 11 & Sec. 10.11 Ewens-Grant, Chps. 4, 7, 11 c 2012 Aybar C. Acar Including slides by Tolga Can Rev. 1.2 (Build 20130528191100) BIN 504 - Bayes

### Online EFFECTIVE AS OF JANUARY 2013

2013 A and C Session Start Dates (A-B Quarter Sequence*) 2013 B and D Session Start Dates (B-A Quarter Sequence*) Quarter 5 2012 1205A&C Begins November 5, 2012 1205A Ends December 9, 2012 Session Break

### Math 018 Review Sheet v.3

Math 018 Review Sheet v.3 Tyrone Crisp Spring 007 1.1 - Slopes and Equations of Lines Slopes: Find slopes of lines using the slope formula m y y 1 x x 1. Positive slope the line slopes up to the right.

### 1.4. Arithmetic of Algebraic Fractions. Introduction. Prerequisites. Learning Outcomes

Arithmetic of Algebraic Fractions 1.4 Introduction Just as one whole number divided by another is called a numerical fraction, so one algebraic expression divided by another is known as an algebraic fraction.

### Points Addressed in this Lecture. Standard form of Boolean Expressions. Lecture 5: Logic Simplication & Karnaugh Map

Points Addressed in this Lecture Lecture 5: Logic Simplication & Karnaugh Map Professor Peter Cheung Department of EEE, Imperial College London (Floyd 4.5-4.) (Tocci 4.-4.5) Standard form of Boolean Expressions

### Math 141. Lecture 1: Conditional Probability. Albyn Jones 1. 1 Library jones/courses/141

Math 141 Lecture 1: Conditional Probability Albyn Jones 1 1 Library 304 jones@reed.edu www.people.reed.edu/ jones/courses/141 Last Time Definitions: Sample Space, Events Last Time Definitions: Sample Space,

### Chapter Prove or disprove: A (B C) = (A B) (A C). Ans: True, since

Chapter 2 1. Prove or disprove: A (B C) = (A B) (A C)., since A ( B C) = A B C = A ( B C) = ( A B) ( A C) = ( A B) ( A C). 2. Prove that A B= A B by giving a containment proof (that is, prove that the

### Week 2 Polygon Triangulation

Week 2 Polygon Triangulation What is a polygon? Last week A polygonal chain is a connected series of line segments A closed polygonal chain is a polygonal chain, such that there is also a line segment

### Compiler Design. Spring Control-Flow Analysis. Sample Exercises and Solutions. Prof. Pedro C. Diniz

Compiler Design Spring 2010 Control-Flow Analysis Sample Exercises and Solutions Prof. Pedro C. Diniz USC / Information Sciences Institute 4676 Admiralty Way, Suite 1001 Marina del Rey, California 90292

### Functional Dependencies: Part 2

Yufei Tao Department of Computer Science and Engineering Chinese University of Hong Kong In designing a database, for the purpose of minimizing redundancy, we need to collect a set F of functional dependencies

### Methods for Constructing Balanced Elimination Trees and Other Recursive Decompositions

Methods for Constructing Balanced Elimination Trees and Other Recursive Decompositions Kevin Grant and Michael C. Horsch Dept. of Computer Science University of Saskatchewan Saskatoon, Saskatchewan, Canada

### Reasoning Under Uncertainty: Variable Elimination Example

Reasoning Under Uncertainty: Variable Elimination Example CPSC 322 Lecture 29 March 26, 2007 Textbook 9.5 Reasoning Under Uncertainty: Variable Elimination Example CPSC 322 Lecture 29, Slide 1 Lecture

### The intuitions in examples from last time give us D-separation and inference in belief networks

CSC384: Lecture 11 Variable Elimination Last time The intuitions in examples from last time give us D-separation and inference in belief networks a simple inference algorithm for networks without Today

### p if x = 1 1 p if x = 0

Probability distributions Bernoulli distribution Two possible values (outcomes): 1 (success), 0 (failure). Parameters: p probability of success. Probability mass function: P (x; p) = { p if x = 1 1 p if

### Definition. A graph is a collection of vertices, and edges between them. They are often represented by a drawing:

1. GRAPHS AND COLORINGS Definition. A graph is a collection of vertices, and edges between them. They are often represented by a drawing: 3 vertices 3 edges 4 vertices 4 edges 4 vertices 6 edges A graph

### Bayesian Networks. Read R&N Ch. 14.1-14.2. Next lecture: Read R&N 18.1-18.4

Bayesian Networks Read R&N Ch. 14.1-14.2 Next lecture: Read R&N 18.1-18.4 You will be expected to know Basic concepts and vocabulary of Bayesian networks. Nodes represent random variables. Directed arcs

### STAT 535 Lecture 4 Graphical representations of conditional independence Part II Bayesian Networks c Marina Meilă

TT 535 Lecture 4 Graphical representations of conditional independence Part II Bayesian Networks c Marina Meilă mmp@stat.washington.edu 1 limitation of Markov networks Between two variables or subsets

### 1. True or False? A voltage level in the range 0 to 2 volts is interpreted as a binary 1.

File: chap04, Chapter 04 1. True or False? A voltage level in the range 0 to 2 volts is interpreted as a binary 1. 2. True or False? A gate is a device that accepts a single input signal and produces one

### United States Naval Academy Electrical and Computer Engineering Department. EC262 Exam 1

United States Naval Academy Electrical and Computer Engineering Department EC262 Exam 29 September 2. Do a page check now. You should have pages (cover & questions). 2. Read all problems in their entirety.

### December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B. KITCHENS

December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B KITCHENS The equation 1 Lines in two-dimensional space (1) 2x y = 3 describes a line in two-dimensional space The coefficients of x and y in the equation

### Project and Production Management Prof. Arun Kanda Department of Mechanical Engineering Indian Institute of Technology, Delhi

Project and Production Management Prof. Arun Kanda Department of Mechanical Engineering Indian Institute of Technology, Delhi Lecture - 9 Basic Scheduling with A-O-A Networks Today we are going to be talking

### Lecture 2: The Complexity of Some Problems

IAS/PCMI Summer Session 2000 Clay Mathematics Undergraduate Program Basic Course on Computational Complexity Lecture 2: The Complexity of Some Problems David Mix Barrington and Alexis Maciel July 18, 2000

### Boolean Algebra Part 1

Boolean Algebra Part 1 Page 1 Boolean Algebra Objectives Understand Basic Boolean Algebra Relate Boolean Algebra to Logic Networks Prove Laws using Truth Tables Understand and Use First Basic Theorems

### On Graph Powers for Leaf-Labeled Trees

Journal of Algorithms 42, 69 108 (2002) doi:10.1006/jagm.2001.1195, available online at http://www.idealibrary.com on On Graph Powers for Leaf-Labeled Trees Naomi Nishimura 1 and Prabhakar Ragde 2 Department

### Mining Social Network Graphs

Mining Social Network Graphs Debapriyo Majumdar Data Mining Fall 2014 Indian Statistical Institute Kolkata November 13, 17, 2014 Social Network No introduc+on required Really? We s7ll need to understand

### 5 Directed acyclic graphs

5 Directed acyclic graphs (5.1) Introduction In many statistical studies we have prior knowledge about a temporal or causal ordering of the variables. In this chapter we will use directed graphs to incorporate

### Lesson 3. Algebraic graph theory. Sergio Barbarossa. Rome - February 2010

Lesson 3 Algebraic graph theory Sergio Barbarossa Basic notions Definition: A directed graph (or digraph) composed by a set of vertices and a set of edges We adopt the convention that the information flows

### 6 Proportion: Fractions, Direct and Inverse Variation, and Percent

6 Proportion: Fractions, Direct and Inverse Variation, and Percent 6.1 Fractions Every rational number can be written as a fraction, that is a quotient of two integers, where the divisor of course cannot

### Incorporating Evidence in Bayesian networks with the Select Operator

Incorporating Evidence in Bayesian networks with the Select Operator C.J. Butz and F. Fang Department of Computer Science, University of Regina Regina, Saskatchewan, Canada SAS 0A2 {butz, fang11fa}@cs.uregina.ca

### Students in their first advanced mathematics classes are often surprised

CHAPTER 8 Proofs Involving Sets Students in their first advanced mathematics classes are often surprised by the extensive role that sets play and by the fact that most of the proofs they encounter are

### Graphical Models. CS 343: Artificial Intelligence Bayesian Networks. Raymond J. Mooney. Conditional Probability Tables.

Graphical Models CS 343: Artificial Intelligence Bayesian Networks Raymond J. Mooney University of Texas at Austin 1 If no assumption of independence is made, then an exponential number of parameters must

### Andrew Blake and Pushmeet Kohli

1 Introduction to Markov Random Fields Andrew Blake and Pushmeet Kohli This book sets out to demonstrate the power of the Markov random field (MRF) in vision. It treats the MRF both as a tool for modeling

### Graph. Consider a graph, G in Fig Then the vertex V and edge E can be represented as:

Graph A graph G consist of 1. Set of vertices V (called nodes), (V = {v1, v2, v3, v4...}) and 2. Set of edges E (i.e., E {e1, e2, e3...cm} A graph can be represents as G = (V, E), where V is a finite and

### 1 Lecture: Integration of rational functions by decomposition

Lecture: Integration of rational functions by decomposition into partial fractions Recognize and integrate basic rational functions, except when the denominator is a power of an irreducible quadratic.

PYTHAGOREAN TRIPLES KEITH CONRAD 1. Introduction A Pythagorean triple is a triple of positive integers (a, b, c) where a + b = c. Examples include (3, 4, 5), (5, 1, 13), and (8, 15, 17). Below is an ancient

### THREE DIMENSIONAL GEOMETRY

Chapter 8 THREE DIMENSIONAL GEOMETRY 8.1 Introduction In this chapter we present a vector algebra approach to three dimensional geometry. The aim is to present standard properties of lines and planes,

### Covariance and Correlation

Covariance and Correlation ( c Robert J. Serfling Not for reproduction or distribution) We have seen how to summarize a data-based relative frequency distribution by measures of location and spread, such

### Eigenvalues and Markov Chains

Eigenvalues and Markov Chains Will Perkins April 15, 2013 The Metropolis Algorithm Say we want to sample from a different distribution, not necessarily uniform. Can we change the transition rates in such

### DATA ANALYSIS II. Matrix Algorithms

DATA ANALYSIS II Matrix Algorithms Similarity Matrix Given a dataset D = {x i }, i=1,..,n consisting of n points in R d, let A denote the n n symmetric similarity matrix between the points, given as where

### 1 Point Inclusion in a Polygon

Comp 163: Computational Geometry Tufts University, Spring 2005 Professor Diane Souvaine Scribe: Ryan Coleman Point Inclusion and Processing Simple Polygons into Monotone Regions 1 Point Inclusion in a

### Tutorial on variational approximation methods. Tommi S. Jaakkola MIT AI Lab

Tutorial on variational approximation methods Tommi S. Jaakkola MIT AI Lab tommi@ai.mit.edu Tutorial topics A bit of history Examples of variational methods A brief intro to graphical models Variational

### Outline. NP-completeness. When is a problem easy? When is a problem hard? Today. Euler Circuits

Outline NP-completeness Examples of Easy vs. Hard problems Euler circuit vs. Hamiltonian circuit Shortest Path vs. Longest Path 2-pairs sum vs. general Subset Sum Reducing one problem to another Clique

### 4.2 Algebraic Properties: Combining Expressions

4.2 Algebraic Properties: Combining Expressions We begin this section with a summary of the algebraic properties of numbers. Property Name Property Example Commutative property (of addition) Commutative

### PUTNAM TRAINING POLYNOMIALS. Exercises 1. Find a polynomial with integral coefficients whose zeros include 2 + 5.

PUTNAM TRAINING POLYNOMIALS (Last updated: November 17, 2015) Remark. This is a list of exercises on polynomials. Miguel A. Lerma Exercises 1. Find a polynomial with integral coefficients whose zeros include

### CHAPTER 5 WELFARE ANALYSIS Microeconomics in Context (Goodwin, et al.), 3 rd Edition

CHAPTER 5 WELFARE ANALYSIS Microeconomics in Context (Goodwin, et al.), 3 rd Edition Chapter Overview This chapter presents welfare analysis, including the topics of consumer and producer surplus. This

### DAGs, I-Maps, Factorization, d-separation, Minimal I-Maps, Bayesian Networks. Slides by Nir Friedman

DAGs, I-Maps, Factorization, d-separation, Minimal I-Maps, Bayesian Networks Slides by Nir Friedman. Probability Distributions Let X 1,,X n be random variables Let P be a joint distribution over X 1,,X

### Notes on Matrix Multiplication and the Transitive Closure

ICS 6D Due: Wednesday, February 25, 2015 Instructor: Sandy Irani Notes on Matrix Multiplication and the Transitive Closure An n m matrix over a set S is an array of elements from S with n rows and m columns.

### Theorem 2. If x Q and y R \ Q, then. (a) x + y R \ Q, and. (b) xy Q.

Math 305 Fall 011 The Density of Q in R The following two theorems tell us what happens when we add and multiply by rational numbers. For the first one, we see that if we add or multiply two rational numbers

### The Sealed Bid Auction Experiment:

The Sealed Bid Auction Experiment: This is an experiment in the economics of decision making. The instructions are simple, and if you follow them carefully and make good decisions, you may earn a considerable

### CMSC 451: Graph Properties, DFS, BFS, etc.

CMSC 451: Graph Properties, DFS, BFS, etc. Slides By: Carl Kingsford Department of Computer Science University of Maryland, College Park Based on Chapter 3 of Algorithm Design by Kleinberg & Tardos. Graphs

### Homework 15 Solutions

PROBLEM ONE (Trees) Homework 15 Solutions 1. Recall the definition of a tree: a tree is a connected, undirected graph which has no cycles. Which of the following definitions are equivalent to this definition

### Querying Joint Probability Distributions

Querying Joint Probability Distributions Sargur Srihari srihari@cedar.buffalo.edu 1 Queries of Interest Probabilistic Graphical Models (BNs and MNs) represent joint probability distributions over multiple

### Lecture 6: Approximation via LP Rounding

Lecture 6: Approximation via LP Rounding Let G = (V, E) be an (undirected) graph. A subset C V is called a vertex cover for G if for every edge (v i, v j ) E we have v i C or v j C (or both). In other

### CPM Educational Program

CPM Educational Program A California, Non-Profit Corporation Chris Mikles, National Director (888) 808-4276 e-mail: mikles @cpm.org CPM Courses and Their Core Threads Each course is built around a few

### Data Structure [Question Bank]

Unit I (Analysis of Algorithms) 1. What are algorithms and how they are useful? 2. Describe the factor on best algorithms depends on? 3. Differentiate: Correct & Incorrect Algorithms? 4. Write short note:

### Determining If Two Graphs Are Isomorphic 1

Determining If Two Graphs Are Isomorphic 1 Given two graphs, it is often really hard to tell if they ARE isomorphic, but usually easier to see if they ARE NOT isomorphic. Here is our first idea to help

### Exponential Random Graph Models for Social Network Analysis. Danny Wyatt 590AI March 6, 2009

Exponential Random Graph Models for Social Network Analysis Danny Wyatt 590AI March 6, 2009 Traditional Social Network Analysis Covered by Eytan Traditional SNA uses descriptive statistics Path lengths

### Data Structures in Java. Session 16 Instructor: Bert Huang

Data Structures in Java Session 16 Instructor: Bert Huang http://www.cs.columbia.edu/~bert/courses/3134 Announcements Homework 4 due next class Remaining grades: hw4, hw5, hw6 25% Final exam 30% Midterm

### Chapter 4. Combinational Logic. Outline. ! Combinational Circuits. ! Analysis and Design Procedures. ! Binary Adders. ! Other Arithmetic Circuits

Chapter 4 Combinational Logic 4- Outline! Combinational Circuits! Analysis and Design Procedures! Binary Adders! Other Arithmetic Circuits! Decoders and Encoders! Multiplexers 4-2 Combinational v.s Sequential

### Discrete mathematics is the study of techniques, ideas and modes

CHAPTER 1 Discrete Systems Discrete mathematics is the study of techniques, ideas and modes of reasoning that are indispensable in applied disciplines such as computer science or information technology.

### Lecture 2: Introduction to belief (Bayesian) networks

Lecture 2: Introduction to belief (Bayesian) networks Conditional independence What is a belief network? Independence maps (I-maps) January 7, 2008 1 COMP-526 Lecture 2 Recall from last time: Conditional