Problem Set 4: Covariance functions and Gaussian random fields

Transcription

1 Problem Set : Covariance functions and Gaussian random fields GEOS 7: Inverse Problems and Parameter Estimation, Carl Tape Assigned: February 9, 15 Due: February 1, 15 Last compiled: February 5, 15 Overview and instructions 1. This problem set deals with three well-known probability distributions: the uniform distribution, the exponential distribution, and the Gaussian distribution.. Reading from Tarantola (5): Ch. (note Example.1) and Sections 5.1, 5., 5.3 (note 5.3.3). 3. Reading from Aster et al. (1): Appendix B. See class notes! 5. (svn update) cp covrand_template.m covrand.m Problem 1 (1.). Gaussian and exponential PDFs 1. (.3) Consider the exponential probability density function ( ) x µ f(x) = k exp σ (1) Show that k = 1/(σ ). Hint: Split the integration into two intervals in order to eliminate the absolute values.. (.5) Consider the two Gaussian probability density functions f X (x) = k x exp ( (x µ x) ) σx f Y (y) = k y exp ( (y µ y) ) σ y () (3) (a) Assuming that the variables X and Y are independent, what is the joint probability density function f(x,y)? (b) Assuming that the mean is zero and that f(x,y) has circular level surfaces, show that the normalization factor for f(x,y) is h = 1/(πσ ) (such that f(x,y) = hg(x,y)). Hints: Aster et al. (1, eq. B.) What does mean zero and circular Gaussian imply about f(x,y)? Try the integration using polar coordinates (it is clean). 3. (.) Make a plot of the standard normal distribution, f N (x). Label σ and µ on your plot. What is the exact expression and approximate value of f N (±σ)? 1

2 Problem (.5). Uniform PDF (and central limit theorem) The formulas for expected value and variance are given by E[X] = x f X (x)dx () Var[X] = E[X ] (E[X]) (5) where f X (x) is a probability density function. The expectated value of g(x) is given by E[g(X)] = g(x) f X (x)dx () 1. (.) Write the expression for a uniform distribution, f U (x), on the interval [a,b]. Write the Matlab command to generate n samples of f U (x).. (1.) Using Equations () () (with your f U (x) in place of f X (x)), show that the expected value and variance for f U (x) are given by E[X] = Var[X] = a + b (b a) 1 (7) () Hint: You will probably need to use polynomial long division. 3. (.) In Matlab, generate 5 or so samples of f U (x) (remember: a sample of f U (x) will be a random number between a and b), and check that the mean (µ) and variance (σ ) of the samples are close to the theoretical values, i.e., µ E[X] and σ Var[X]. For the sake of comparison, use a = 1, b = 5 1 Plot a histogram of your samples to check that the distribution is flat over the appropriate interval. (No need to turn in this plot.). (1.3) The central limit theorem is stated in Aster et al. (1, Section B.): Let X 1, X,..., X n be independent and identically distributed (IID) random variables with a finite expected value µ and variance σ. Let Z n = X 1 + X + + X n nµ σ. (9) n In the limit as n approaches infinity, the distribution of Z n approaches the standard normal distribution. The central limit theorem works for any kind of distribution. You will demonstrate it using the uniform distribution, f U (x) (Problem -1), for which you know µ and σ. (a) (.1) Write the expression for Z 1. What are the minimum and maximum values of Z 1?

3 Hint: Your answer should not have µ or σ in the expressions, since these can be written in terms of a and b. (b) (.) i. Write the expression for Z. ii. What are the minimum and maximum possible values of the sum X 1 + X? iii. What are the minimum and maximum values of Z? Hint: Your answer should not have µ or σ in the expressions, since these can be written in terms of a and b. (c) (.) By generating samples (X) from your f U (x), demonstrate the central limit theorem by showing a set of histograms of Z 1, Z, Z 3, and Z. To obtain each distribution of Z n (Eq. 9), you will need to repeat the experiment p times; try p = 5. Center your histograms between ±. Hint: Consider the case of n =. The first experiment will involve generating two random samples, X 1 and X, of f U (x). You can then compute Z using Equation (9). You then repeat this process p times and plot a histogram of the p values of Z. Problem 3 (3.5). Estimating a covariance matrix from a set of samples See the template script covrand.m. Let P be the number of samples and M be the number of model parameters describing a single sample. The ith sample is represented by the M 1 vector m i. It may (or may not) help to attach some physical meaning to these samples. Think of each sample as the functional variation in a single dimension. The set of samples might represent, for example: the variation in topography along different transects. the variation in height of an interval of an oscillating wire: each profile represents a different time. the variation of vertical ground displacement with time, as captured by a seismogram: each profile is for a different earthquake. In this problem, the goal is to compute a sample covariance matrix, C P, and to use it to estimate the covariance function, C(d), that characterizes the samples. 1. (.) Run covrand.m. Identify the key variables (and their dimensions) that are loaded into Matlab, then comment the break statement and proceed.. (.3) Plot samples in a subplot figure, with one sample per subplot and with the same axis scale for each subplot (use ax from covrand.m). Plot each sample using the spatial discretization given by x. 3. (1.5) Use the first P = samples to do the following: (a) (.3) Compute and plot the mean, µ. 3

4 (b) (.7) Compute and plot the covariance matrix, C (use imagesc). Show your code to compute C, and do not use the black-box cov function 1. (c) (.5) Make a scatterplot of (C ) kk versus D kk = x k x k, where D is provided in covrand.m. Hint: Try plot(d,csamp, b. ); where Csamp represents C.. (.5) Repeat the previous (include plots), but use all samples. How does the estimated mean and covariance change with increasing the number of samples? 5. (.) Examine the script covc.m. Some example plots using covc.m are shown in Figure 1. Two of the functions plotted are ) C gaus (d) = σ exp ( d L () ( C exp (d) = σ exp d ) L, (11) where d is the distance between x and x. In our 1D example, d(x,x ) = x x. Note. C takes in a distance between two points and outputs a value. It can alternatively be written as a function of the two input points, x and x : C gaus (x,x ) = σ exp ( (x x ) ) L (1) ( C exp (x,x ) = σ exp x ) x L, (13) or in discrete form (C gaus ) kk = C gaus (x k,x k ) = σ exp ( (x k x k ) ) L (1) ( (C exp ) kk = C exp (x k,x k ) = σ exp x ) k x k L, (15) (a) What are C gaus and C exp for two points separated by d = L? (b) What are C gaus and C exp for two points separated by d = L /? (c) What are C gaus and C exp for two points separated by d =? (d) What values of σ and L were used for Figure 1? Note: There should be no decimal numbers in your answers.. (.) Run the example listed in covc.m and make sure you understand what the input parameters are. 7. (.5) Use covc.m to find a covariance function, C(d), that reasonably fits the scatterplot of (C ) kk versus D kk from Problem If you use cov to check, you may need to transpose your matrix of samples to ensure that the resultant matrix is M M. I have used the notation L = L to distinguish our L from the L that appears in Tarantola (5).

5 (a) List your values of the parameters that describe C(d). (b) What are the diagonal entries of C and why? (c) Include a plot with C(d) superimposed on the scatterplot of (C ) kk versus D kk. (d) Include a plot of C (use imagesc). Problem (3.). Generating samples from a prescribed covariance Tarantola (5, p. 5):...a large enough number of realizations completely characterizes the [Gaussian random] field...displaying the mean of the Gaussian random field and plotting the covariance is not an alternative to displaying a certain number of realizations, because the mean and covariance do not relate in an intuitive way to the realizations. In Problem 3, you used a set of samples and computed a mean, µ, and a covariance matrix, C. You used C to estimate a covariance function, C(d), with corresponding covariance matrix C. Here you will use µ and C (not C ) to generate a set of samples that (hopefully) resembles the original samples. 1. (1.5) (a) (1.) Generate samples of C, and save these as a set of m C. Include your code. Hint: A = chol(c, lower ); Also, if x = Aw is a sample of C, what is w? (b) (.) Add µ to each m C, then plot the first samples (as in Problem 3-). Superimpose µ in each subplot. (c) (.1) Do your samples resemble those provided in Problem 3? (yes or no). (.5) Consider the samples of the covariance matrix, m C. Compute the mean (mean), standard deviation (std), and norm of each of the m C, and show your results in three histogram plots. Do your results check with what you expect? NOTE: Matlab s norm command will not be useful here. In calculating the norm, you will need to use a modified covariance matrix, M C, where M M is the dimension of C. This will ensure that the norm of each m C is about (.) Now generate a new C usingcovc.m by making only one change: changeicov to either 1 or. Repeat Problem -1 using the same set of Gassian random vectors, w i, as before. This will allow for a true comparison between samples from the Gaussian or exponential covariance functions. (a) Generate samples of the new C, add µ to each sample. Plot the the first samples. (b) Describe the differences and similarities between the samples from the two different distributions.. (.) Repeat Problem - for the set of samples from the new C. Problem Approximately how many hours did you spend on this problem set? Feel free to suggest improvements here. 5

6 Gaussian covariance Exponential covariance exp(.5) exp(.71) exp( 1.) exp( 1.1) exp(.) Circular covariance Matern covariance nu =.,.5, 1.5, Figure 1: Covariance functions from covc.m characterized by length scale L and amplitude σ. See Tarantola (5, Section 5.3.3, p. 113). Some reference e-folding depths are labeled; for example, the y-values of the top line is y = σ e 1/ 9.7. The Matérn covariance functions include an additional parameter, ν, that influences the shape: ν for the Gaussian function (upper left), ν =.5 for the exponential function (upper right).

7 References Aster, R. C., B. Borchers, and C. H. Thurber (1), Parameter Estimation and Inverse Problems, ed., Elsevier, Waltham, Mass., USA. Tarantola, A. (5), Inverse Problem Theory and Methods for Model Parameter Estimation, SIAM, Philadelphia, Penn., USA. 7