Computational Foundations of Cognitive Science

Transcription

1 Computational Foundations of Cognitive Science Lecture 15: Convolutions and Kernels Frank Keller School of Informatics University of Edinburgh February 23, 2010 Frank Keller Computational Foundations of Cognitive Science 1

2 1 Convolutions of Discrete Functions 2 Frank Keller Computational Foundations of Cognitive Science 2

3 Convolutions of Discrete Functions : Convolution If f and g are discrete functions, then f g is the convolution of f and g and is defined as: (f g)(x) = + u= f (u)g(x u) Intuitively, the convolution of two functions represents the amount of overlap between the two functions. The function g is the input, f the kernel of the convolution. Convolutions are often used for filtering, both in the temporal or frequency domain (one dimensional) and in the spatial domain (two dimensional). Frank Keller Computational Foundations of Cognitive Science 3

4 Convolutions of Discrete Functions Theorem: Properties of Convolution If f, g, and h are functions and a is a constant, then: f g = g f (commutativity) f (g h) = (f g) h (associativity) f (g + h) = (f g) + (f h) (distributivity) a(f g) = (af ) g = f (ag) (associativity with scalar multiplication) Note that it doesn t matter if g or f is the kernel, due to commutativity. Frank Keller Computational Foundations of Cognitive Science 4

5 If a function f ranges over a finite set of values a = a 1, a 2,..., a n, then it can be represented as vector [ a 1 a 2... a n ]. : If the functions f and g are represented as vectors a = [ a 1 a 2... a m ] and b = [ b1 b 2... b n ], then f g is a vector c = [ c 1 c 2... c m+n 1 ] as follows: c x = u a u b x u+1 where u ranges over all legal subscripts for a u and b x u+1, specifically u = max(1, x n + 1)... min(x, m). Frank Keller Computational Foundations of Cognitive Science 5

6 If we assume that the two vectors a and b have the same dimensionality, then the convolution c is: c 1 = a 1 b 1 c 2 = a 1 b 2 + a 2 b 1 c 3 = a 1 b 3 + a 2 b 2 + a 3 b 1... c n = a 1 b n + a 2 b n a n b 1... c 2n 1 = a n b n Note that the sum for each component only includes those products for which the subscripts are valid. Frank Keller Computational Foundations of Cognitive Science 6

7 Example 1 0 Assume a = 0, b = Then a 1 b a 1 b 2 + a 2 b 1 a b = a 1 b 3 + a 2 b 2 + a 3 b 1 a 2 b 3 + a 3 b 2 = ( 1) ( 1)1 = a 3 b 3 ( 1) Frank Keller Computational Foundations of Cognitive Science 7

8 What happens if the kernel is smaller than the input vector (this is actually the typical case)? Assume a = [ ] 1 2 1, b = Compute a b. 1 1 What is the purpose of the kernel a? Frank Keller Computational Foundations of Cognitive Science 8

9 We can extend convolution to functions of two variables f (x, y) and g(x, y). : Convolution for Functions of two Variables If f and g are discrete functions of two variables, then f g is the convolution of f and g and is defined as: (f g)(x, y) = + + u= v= f (u, v)g(x u, y v) Frank Keller Computational Foundations of Cognitive Science 9

10 We can regard functions of two variables as matrices with A xy = f (x, y), and obtain a matrix definition of convolution. : If the functions f and g are represented as the n m matrix A and the k l matrix B, then f g is an (n + k 1) (m + l 1) matrix C: c xy = a uv b x u+1,y v+1 u v where u and v range over all legal subscripts for a uv and b x u+1,y v+1. Note: the treatment of subscripts can vary from implementation to implementation, and affects the size of C (this is parameterizable in Matlab, see documentation of conv2 function). Frank Keller Computational Foundations of Cognitive Science 10

11 Example b a 11 a 12 a 11 b 12 b 13 b 14 b Let A = a 21 a 22 a 23 and B = b 21 b 22 b 23 b 24 b 25 b a 31 a 32 a 31 b 32 b 33 b 34 b b 41 b 42 b 43 b 44 b 45 Then for C = A B, the entry c 33 = a 11 b 33 + a 12 b 32 + a 13 b 31 + a 21 b 23 + a 22 b 22 + a 23 b 21 + a 31 b 13 + a 32 b 12 + a 33 b 11. Here, B could represent an image, and A could represent a kernel performing an image operation, for instance. Frank Keller Computational Foundations of Cognitive Science 11

12 Example: Image Processing Convolving and image with a kernel (typically a 3 3 matrix) is a powerful tool for image processing. B = K B = 1/9 1/9 1/9 K = 1/9 1/9 1/9 implements a mean filter which smooths an 1/9 1/9 1/9 image by replacing each pixel value with the mean of its neighbors. Frank Keller Computational Foundations of Cognitive Science 12

13 Example: Image Processing B = K B = The kernel K = implements the Sobel edge detector It detects gradients in the pixel values (sudden changes in brightness), which correspond to edges. The example is for vertical edges. Frank Keller Computational Foundations of Cognitive Science 13

14 We can also define convolution for continuous functions. In this case, we replace the sums by integrals in the definition. : Convolution If f and g are continuous functions, then f g is the convolution of f and g and is defined as: (f g)(x) = + f (u)g(x u)du Convolutions of continuous functions are widely used in signal processing for filtering continuous signals, e.g., speech. Frank Keller Computational Foundations of Cognitive Science 14

15 Example Assume{ the following step functions: 3 if 0 x 4 g(x) = f (x) = 0 otherwise If we integrate g(x), we get: G(x) = Then the convolution f g is: (f g)(x) = + f (u)g(x u)du = x 1 2 x+1 g(u)du = 1 2 (G(x 1) G(x + 1)) = 3 2 (x + 1) if 1 x < 1 3 if 1 x (x 1) + 6 if 3 < x 5 0 otherwise { 1 2 if 1 x 1 0 otherwise 0 if x 0 3x if 0 x if x > 4 1 g(x u)du = 1 Frank Keller Computational Foundations of Cognitive Science 15

16 Function g(x) (blue) and convolution f g(x) (green): Frank Keller Computational Foundations of Cognitive Science 16

17 Assume we have a function I (x) that represents the intensity of a signal over time. This can be a very spiky function. We can make this function less spiky by convolving it with a Gaussian kernel. This is a kernel given by the Gaussian function: G(x) = 1 2π e 1 2 x2 The convolution G I is a smoothed version of the original intensity function. We will learn more about the Gaussian function (aka normal distribution) in the second half of this course. Frank Keller Computational Foundations of Cognitive Science 17

18 Gaussian function G(x): Frank Keller Computational Foundations of Cognitive Science 18

19 Original intensity function I (x): Frank Keller Computational Foundations of Cognitive Science 19

20 Smoothed function obtained by convolution with a Gaussian kernel: Frank Keller Computational Foundations of Cognitive Science 20

21 Summary The convolution (f g)(x) = f (u)g(x u) represents the overlap between a discrete function g and a kernel f ; convolutions in one dimension can be represented as vectors, convolutions in two dimensions as matrices; in image processing, two dimensional convolution can be used to filter an image or for edge detection; for continuous functions, convolution is defined as (f g)(x) = f (u)g(x u)du; this can be used in signal processing, e.g., to smooth a signal. Frank Keller Computational Foundations of Cognitive Science 21