Max Flow. Lecture 4. Optimization on graphs. C25 Optimization Hilary 2013 A. Zisserman. Max-flow & min-cut. The augmented path algorithm

Transcription

1 Lecture 4 C5 Optimization Hilary 03 A. Zisserman Optimization on graphs Max-flow & min-cut The augmented path algorithm Optimization for binary image graphs Applications Max Flow Given: a weighted directed graph with two distinguished nodes: source s, sink (destination) t B reminder Interpret edge weights (all positive) as capacities Goal: Find maximum flow from s to t Flow does not exceed capacity in any edge Flow at every vertex satisfies equilibrium [ flow in equals flow out ] e.g. oil flowing through pipes, internet routing

2 Example 3 cont. B reminder Slide: Robert Sedgewick and Kevin Wayne Example cont. B reminder

3 Max flow example 3 S T max flow is unique = but there may be multiple paths (solutions) that achieve it Matlab LP function linprog for max-flow >> f = [ -; -; 0; 0; 0 ]; >> A = [ e e e 3 e 4 e ]; >> b = [; ; ; 3; ]; >> Aeq = [ >> beq = [ 0; 0 ]; >> lb = zeros(5,); ]; >> x = linprog( f, A, b, Aeq, beq, lb ); 3 e e 4 S e 3 T e e 5 >> Optimization terminated. >> x x = >> flow = - x * f flow =.0

4 Another example 9 / 5/9 / v v 5 4 v v 3/5 3 4/4 Max Flow = 7 The st-mincut Problem An st-cut (S,T) divides the nodes between source and sink 9 The cost of the cut is the sum of costs of all edges going from S to T v v 5 4 The st-min-cut is the cut with lowest cost Each node is either assigned to the source S or sink T The cost of the edge (i, j) is taken if (i S) and (j T) Slides from Pushmeet Kohli

5 The st-mincut Problem An st-cut (S,T) divides the nodes between source and sink 9 The cost of the cut is the sum of costs of all edges going from S to T v v 5 4 The st-min-cut is the cut with lowest cost Each node is either assigned to the source S or sink T = 6 The cost of the edge (i, j) is taken if (i S) and (j T) The st-mincut Problem An st-cut (S,T) divides the nodes between source and sink 9 The cost of the cut is the sum of costs of all edges going from S to T v v 5 4 The st-min-cut is the cut with lowest cost Each node is either assigned to the source S or sink T = 7 The cost of the edge (i, j) is taken if (i S) and (j T)

6 Min-cut\Max-flow Theorem 9 v v 5 4 In every network, the maximum flow equals the cost of the st-mincut Max flow = min cut = 7 Next: the augmented path algorithm for computing the max-flow/min-cut Maxflow Algorithms v v 5 Flow = Augmenting Path Based Algorithms. Find path from source to sink with positive capacity. Push maximum possible flow through this path 3. Repeat until no path can be found Algorithms assume non-negative capacity

7 Maxflow Algorithms v v 5 Flow = Augmenting Path Based Algorithms. Find path from source to sink with positive capacity. Push maximum possible flow through this path 3. Repeat until no path can be found Algorithms assume non-negative capacity Maxflow Algorithms - 9 v v 5- Flow = Augmenting Path Based Algorithms. Find path from source to sink with positive capacity. Push maximum possible flow through this path 3. Repeat until no path can be found Algorithms assume non-negative capacity

8 Maxflow Algorithms 0 9 v v 3 Flow = 4 Augmenting Path Based Algorithms. Find path from source to sink with positive capacity. Push maximum possible flow through this path 3. Repeat until no path can be found Algorithms assume non-negative capacity Maxflow Algorithms 0 9 v v 3 Flow = 4 Augmenting Path Based Algorithms. Find path from source to sink with positive capacity. Push maximum possible flow through this path 3. Repeat until no path can be found Algorithms assume non-negative capacity

9 Maxflow Algorithms 0 9 v v 3 Flow = 4 Augmenting Path Based Algorithms. Find path from source to sink with positive capacity. Push maximum possible flow through this path 3. Repeat until no path can be found Algorithms assume non-negative capacity Maxflow Algorithms 0 5 v v 3 Flow = Augmenting Path Based Algorithms. Find path from source to sink with positive capacity. Push maximum possible flow through this path 3. Repeat until no path can be found Algorithms assume non-negative capacity

10 Maxflow Algorithms 0 5 v v 3 Flow = 6 0 Augmenting Path Based Algorithms. Find path from source to sink with positive capacity. Push maximum possible flow through this path 3. Repeat until no path can be found Algorithms assume non-negative capacity Maxflow Algorithms 0 5 v v 3 Flow = 6 0 Augmenting Path Based Algorithms. Find path from source to sink with positive capacity. Push maximum possible flow through this path 3. Repeat until no path can be found Algorithms assume non-negative capacity

11 Maxflow Algorithms v v Flow = Augmenting Path Based Algorithms. Find path from source to sink with positive capacity. Push maximum possible flow through this path 3. Repeat until no path can be found Algorithms assume non-negative capacity Maxflow Algorithms v v Flow = Augmenting Path Based Algorithms. Find path from source to sink with positive capacity. Push maximum possible flow through this path 3. Repeat until no path can be found Algorithms assume non-negative capacity

12 Maxflow Algorithms Flow = 7 Min cut = v v 3 0 v v = 7 Image Graphs many loops very large number of nodes (variables) millions dynamic programming can t be used Consider binary graphs (h = ) pixels

13 Example: noise removal in an image N (i) i j n is the number of pixels in the image, e.g. n = M Graph structure: each vertex is connected to four neighbours noisy data z i = x i + w i where x i {0, } w i N(0, σ ) and φ(x i,x j ) = ( 0 ifxi = x j ifx i 6= x j. 0 i f(x) = X i (z i x i ) +dφ(x i,x i ) Graph cut algorithms Binary optimization: each variable x has one of two possible values f(x) = nx {m i (x i )+ X φ i (x i,x j )} i= j N (i) x i {0, } N (i) is the neighbourhood of node i i The cost function f(x) issub-modularif φ(0, 0) + φ(, ) <= φ(0, ) + φ(, 0) If f(x) is sub-modular, then it can be optimized by the Min-Cut algorithm Complexity of minimization: exhaustive search O( n ) min cut O(n 3 )

14 d = 0 original original plus noise Min x d = 60 original original plus noise Min x n 0 K

15 Optimization using graph cuts Stage : map the cost function f(x) onto a flow network so that a cut of the network corresponds to the cost f(x) Stage : compute the min-cut of the network using an augmented path algorithm Map f(x) onto network flow Construct a network so that a cut corresponds to an assignment of x i m () D Label m () C B Label 0 m (0) m A (0) s n-links w pq = 0 a cut x x t = f(x) = nx {m i (x i )+ X φ i (x i,x j )} i= j N (i)

16 m () D m () C B m (0) m A (0) x x For unary terms only: m () x m (0) (0) m () m (0) () x (0) m () m () cut x =,x = x x m (0) m (0) f(x) =m () + m () () Now, include pair wise term D m () m () C B m (0) m (0) A (0) m () + C - A m () + D - C B+C-A-D x x x x m (0) m (0) x () x 0 A C B D = A + B+C D-C 0 0 -A-D + + C-A C-A 0 D-C 0 0 add C-A if x = add D-C if x = Sub-modular constraint: flows must be positive. So, B+C-A-D >= 0

17 m () m (0) C D A B x x (0) m () + C - A m () + D - C cut B+C-A-D x x x =,x = m (0) m (0) () f(x) =m () + m () + D A m () m (0) C D A B x x (0) m () + C - A m () + D - C cut x =0,x = B+C-A-D x x f(x) = m (0) + m () m (0) m (0) () +D C + B + C A D = m (0) + m () + B A

18 Summary: optimization using graph cuts Stage : map the cost function f(x) onto a flow network so that a cut of the network corresponds to the cost f(x) Stage : compute the min-cut of the network using an augmented path algorithm Applications Optimization of binary image graph using graph-cuts:. Image cut-out and editing. Image quilting 3. Interactive Digital Photo-montage

19 . Image cut-out by binary segmentation Object - white, Background - green/grey Graph G = (V,E) Each vertex corresponds to a pixel Edges define a 4-neighbourhood grid graph Assign a label to each vertex from L = {obj,bkg}

20 Object - white, Background - green/grey Cost of a labelling f : V L Cost of label obj low Graph G = (V,E) Per Vertex Cost Cost of label bkg high Object - white, Background - green/grey Cost of a labelling f : V L Graph G = (V,E) Per Vertex Cost Cost of label obj high UNARY COST Cost of label bkg low

21 Object - white, Background - green/grey Cost of a labelling f : V L Graph G = (V,E) Per Edge Cost Cost of same label low Cost of different labels high Object - white, Background - green/grey Cost of a labelling f : V L Graph G = (V,E) Per Edge Cost Cost of different labels low PAIRWISE COST

22 Object - white, Background - green/grey Graph G = (V,E) Problem: Find the labelling with minimum cost f* N (i) i j f(x) = nx {m i (x i )+ X φ i (x i,x j )} i= j N (i) x i = for foreground pixels, x i = 0 for background m i (x i ) is likelihood that pixel at i is foreground (if x i =),orbackground (if x i = 0 ), e.g. using colour histogram of seed regions φ(x i,x j ) penalizes a change of state: φ i (x i,x j ) = ( 0 ifxi = x j γe β(i i I j ) ifx i 6= x j.

23 Application: foreground/background image segmentation foreground Seed Pixels Background Seed Pixels use seed pixels to learn colour distribution Image editing Available in Microsoft Office

24 Image Quilting Example: Texture Synthesis Goal of Texture Synthesis: create new samples of a given texture Many applications: virtual environments, hole-filling, texturing surfaces

25 block Input texture B B B B B B Random placement of blocks Neighboring blocks constrained by overlap Minimal error boundary cut Algorithm Pick size of block and size of overlap Synthesize blocks in raster order Search input texture for block that satisfies overlap constraints (above and left) Paste new block into resulting texture > use graph cuts to compute minimal error boundary cut Efros & Freeman 00, Kwatra et al. 003

26 Minimal error boundary overlapping blocks vertical boundary _ = overlap error min. error boundary

27 Interactive Digital Photomontage

28 Agarwala et al. 004 Use graph-cuts to quilt images