MATH 590: Meshfree Methods Chapter 7: Conditionally Positive Definite Functions Greg Fasshauer Department of Applied Mathematics Illinois Institute of Technology Fall 2010 fasshauer@iit.edu MATH 590 Chapter 7 1
Outline 1 Conditionally Positive Definite Functions Defined 2 CPD Functions and Generalized Fourier Transforms fasshauer@iit.edu MATH 590 Chapter 7 2
Outline Conditionally Positive Definite Functions Defined 1 Conditionally Positive Definite Functions Defined 2 CPD Functions and Generalized Fourier Transforms fasshauer@iit.edu MATH 590 Chapter 7 3
Conditionally Positive Definite Functions Defined In this chapter we generalize positive definite functions to conditionally positive definite and strictly conditionally positive definite functions of order m. These functions provide a natural generalization of RBF interpolation with polynomial reproduction. Examples of strictly conditionally positive definite (radial) functions are given in the next chapter. fasshauer@iit.edu MATH 590 Chapter 7 4
Conditionally Positive Definite Functions Defined Definition A complex-valued continuous function Φ is called conditionally positive definite of order m on R s if N j=1 k=1 N c j c k Φ(x j x k ) 0 (1) for any N pairwise distinct points x 1,..., x N R s, and c = [c 1,..., c N ] T C N satisfying N c j p(x j ) = 0, j=1 for any complex-valued polynomial p of degree at most m 1. The function Φ is called strictly conditionally positive definite of order m on R s if the quadratic form (1) is zero only for c 0. fasshauer@iit.edu MATH 590 Chapter 7 5
Conditionally Positive Definite Functions Defined An immediate observation is Lemma A function that is (strictly) conditionally positive definite of order m on R s is also (strictly) conditionally positive definite of any higher order. In particular, a (strictly) positive definite function is always (strictly) conditionally positive definite of any order. fasshauer@iit.edu MATH 590 Chapter 7 6
Conditionally Positive Definite Functions Defined An immediate observation is Lemma A function that is (strictly) conditionally positive definite of order m on R s is also (strictly) conditionally positive definite of any higher order. In particular, a (strictly) positive definite function is always (strictly) conditionally positive definite of any order. Proof. The first statement follows immediately from the definition. fasshauer@iit.edu MATH 590 Chapter 7 6
Conditionally Positive Definite Functions Defined An immediate observation is Lemma A function that is (strictly) conditionally positive definite of order m on R s is also (strictly) conditionally positive definite of any higher order. In particular, a (strictly) positive definite function is always (strictly) conditionally positive definite of any order. Proof. The first statement follows immediately from the definition. The second statement is true since (by convention) the case m = 0 yields the class of (strictly) positive definite functions, i.e., (strictly) conditionally positive definite functions of order zero are (strictly) positive definite. fasshauer@iit.edu MATH 590 Chapter 7 6
Conditionally Positive Definite Functions Defined As for positive definite functions we also have (see [Wendland (2005a)] for more details) Theorem A real-valued continuous even function Φ is called conditionally positive definite of order m on R s if N j=1 k=1 N c j c k Φ(x j x k ) 0 (2) for any N pairwise distinct points x 1,..., x N R s, and c = [c 1,..., c N ] T R N satisfying N c j p(x j ) = 0, j=1 for any real-valued polynomial p of degree at most m 1. The function Φ is called strictly conditionally positive definite of order m on R s is zero only for c 0. fasshauer@iit.edu MATH 590 Chapter 7 7
Conditionally Positive Definite Functions Defined Remark If the function Φ is strictly conditionally positive definite of order m, then the matrix A with entries A jk = Φ(x j x k ) can be interpreted as being positive definite on the space of vectors c such that N c j p(x j ) = 0, p Π s m 1. j=1 fasshauer@iit.edu MATH 590 Chapter 7 8
Conditionally Positive Definite Functions Defined Remark If the function Φ is strictly conditionally positive definite of order m, then the matrix A with entries A jk = Φ(x j x k ) can be interpreted as being positive definite on the space of vectors c such that N c j p(x j ) = 0, p Π s m 1. j=1 In this sense A is positive definite on the space of vectors c perpendicular to s-variate polynomials of degree at most m 1. fasshauer@iit.edu MATH 590 Chapter 7 8
Conditionally Positive Definite Functions Defined We can now generalize the theorem we had in the previous chapter for constant precision interpolation to the case of general polynomial reproduction: fasshauer@iit.edu MATH 590 Chapter 7 9
Conditionally Positive Definite Functions Defined We can now generalize the theorem we had in the previous chapter for constant precision interpolation to the case of general polynomial reproduction: Theorem If the real-valued even function Φ is strictly conditionally positive definite of order m on R s and the points x 1,..., x N form an (m 1)-unisolvent set, then the system of linear equations [ A P P T O ] [ c d ] = [ y 0 ] (3) is uniquely solvable. fasshauer@iit.edu MATH 590 Chapter 7 9
Proof Conditionally Positive Definite Functions Defined The proof is almost identical to the proof of the earlier theorem for constant reproduction. fasshauer@iit.edu MATH 590 Chapter 7 10
Proof Conditionally Positive Definite Functions Defined The proof is almost identical to the proof of the earlier theorem for constant reproduction. Assume [c, d] T is a solution of the homogeneous linear system, i.e., with y = 0. fasshauer@iit.edu MATH 590 Chapter 7 10
Proof Conditionally Positive Definite Functions Defined The proof is almost identical to the proof of the earlier theorem for constant reproduction. Assume [c, d] T is a solution of the homogeneous linear system, i.e., with y = 0. We show that [c, d] T = 0 is the only possible solution. fasshauer@iit.edu MATH 590 Chapter 7 10
Proof Conditionally Positive Definite Functions Defined The proof is almost identical to the proof of the earlier theorem for constant reproduction. Assume [c, d] T is a solution of the homogeneous linear system, i.e., with y = 0. We show that [c, d] T = 0 is the only possible solution. Multiplication of the top block of (3) by c T yields c T Ac + c T Pd = 0. fasshauer@iit.edu MATH 590 Chapter 7 10
Proof Conditionally Positive Definite Functions Defined The proof is almost identical to the proof of the earlier theorem for constant reproduction. Assume [c, d] T is a solution of the homogeneous linear system, i.e., with y = 0. We show that [c, d] T = 0 is the only possible solution. Multiplication of the top block of (3) by c T yields c T Ac + c T Pd = 0. From the bottom block of the system we know P T c = 0. This implies c T P = 0 T, and therefore c T Ac = 0. (4) fasshauer@iit.edu MATH 590 Chapter 7 10
Conditionally Positive Definite Functions Defined Since the function Φ is strictly conditionally positive definite of order m by assumption we know that the above quadratic form of A (with coefficients such that P T c = 0) is zero only for c = 0. fasshauer@iit.edu MATH 590 Chapter 7 11
Conditionally Positive Definite Functions Defined Since the function Φ is strictly conditionally positive definite of order m by assumption we know that the above quadratic form of A (with coefficients such that P T c = 0) is zero only for c = 0. Therefore (4) tells us that c = 0. fasshauer@iit.edu MATH 590 Chapter 7 11
Conditionally Positive Definite Functions Defined Since the function Φ is strictly conditionally positive definite of order m by assumption we know that the above quadratic form of A (with coefficients such that P T c = 0) is zero only for c = 0. Therefore (4) tells us that c = 0. The unisolvency of the data sites, i.e., the linear independence of the columns of P (c.f. one of our earlier remarks), and the fact that c = 0 guarantee d = 0 from the top block Ac + Pd = 0 of the homogeneous version of (3). fasshauer@iit.edu MATH 590 Chapter 7 11
Outline CPD Functions and Generalized Fourier Transforms 1 Conditionally Positive Definite Functions Defined 2 CPD Functions and Generalized Fourier Transforms fasshauer@iit.edu MATH 590 Chapter 7 12
CPD Functions and Generalized Fourier Transforms As before, integral characterizations help us identify functions that are strictly conditionally positive definite of order m on R s. fasshauer@iit.edu MATH 590 Chapter 7 13
CPD Functions and Generalized Fourier Transforms As before, integral characterizations help us identify functions that are strictly conditionally positive definite of order m on R s. An integral characterization of conditionally positive definite functions of order m, i.e., a generalization of Bochner s theorem, can be found in the paper [Sun (1993b)]. fasshauer@iit.edu MATH 590 Chapter 7 13
CPD Functions and Generalized Fourier Transforms As before, integral characterizations help us identify functions that are strictly conditionally positive definite of order m on R s. An integral characterization of conditionally positive definite functions of order m, i.e., a generalization of Bochner s theorem, can be found in the paper [Sun (1993b)]. However, since the subject matter is rather complicated, and since it does not really help us solve the scattered data interpolation problem, we do not mention any details here. fasshauer@iit.edu MATH 590 Chapter 7 13
CPD Functions and Generalized Fourier Transforms The Schwartz Space and the Generalized Fourier Transform In order to formulate the Fourier transform characterization of strictly conditionally positive definite functions of order m on R s we require some advanced tools from analysis (see Appendix B). fasshauer@iit.edu MATH 590 Chapter 7 14
CPD Functions and Generalized Fourier Transforms The Schwartz Space and the Generalized Fourier Transform In order to formulate the Fourier transform characterization of strictly conditionally positive definite functions of order m on R s we require some advanced tools from analysis (see Appendix B). First we define the Schwartz space of rapidly decreasing test functions S = {γ C (R s ) : lim x α (D β γ)(x) = 0, α, β N s 0 }, x where we use the multi-index notation D β = x β 1 1 β x βs s, β = s β i. i=1 fasshauer@iit.edu MATH 590 Chapter 7 14
CPD Functions and Generalized Fourier Transforms Some properties of the Schwartz space The Schwartz Space and the Generalized Fourier Transform S consists of all those functions γ C (R s ) which, together with all their derivatives, decay faster than any power of 1/ x. fasshauer@iit.edu MATH 590 Chapter 7 15
CPD Functions and Generalized Fourier Transforms The Schwartz Space and the Generalized Fourier Transform Some properties of the Schwartz space S consists of all those functions γ C (R s ) which, together with all their derivatives, decay faster than any power of 1/ x. S contains the space C 0 (Rs ), the space of all infinitely differentiable functions on R s with compact support. fasshauer@iit.edu MATH 590 Chapter 7 15
CPD Functions and Generalized Fourier Transforms The Schwartz Space and the Generalized Fourier Transform Some properties of the Schwartz space S consists of all those functions γ C (R s ) which, together with all their derivatives, decay faster than any power of 1/ x. S contains the space C 0 (Rs ), the space of all infinitely differentiable functions on R s with compact support. C 0 (Rs ) is a true subspace of S since, e.g., the function γ(x) = e x 2 belongs to S but not to C 0 (Rs ). fasshauer@iit.edu MATH 590 Chapter 7 15
CPD Functions and Generalized Fourier Transforms The Schwartz Space and the Generalized Fourier Transform Some properties of the Schwartz space S consists of all those functions γ C (R s ) which, together with all their derivatives, decay faster than any power of 1/ x. S contains the space C 0 (Rs ), the space of all infinitely differentiable functions on R s with compact support. C 0 (Rs ) is a true subspace of S since, e.g., the function γ(x) = e x 2 belongs to S but not to C 0 (Rs ). γ S has a classical Fourier transform ˆγ which is also in S. fasshauer@iit.edu MATH 590 Chapter 7 15
CPD Functions and Generalized Fourier Transforms The Schwartz Space and the Generalized Fourier Transform Of particular importance are the following subspaces S m of S S m = {γ S : γ(x) = O( x m ) for x 0, m N 0 }. fasshauer@iit.edu MATH 590 Chapter 7 16
CPD Functions and Generalized Fourier Transforms The Schwartz Space and the Generalized Fourier Transform Of particular importance are the following subspaces S m of S S m = {γ S : γ(x) = O( x m ) for x 0, m N 0 }. Furthermore, the set V of slowly increasing functions is given by V = {f C(R s ) : f (x) p(x) for some polynomial p Π s }. fasshauer@iit.edu MATH 590 Chapter 7 16
CPD Functions and Generalized Fourier Transforms The Schwartz Space and the Generalized Fourier Transform The generalized Fourier transform is now given by Definition Let f V be complex-valued. A continuous function ˆf : R s \ {0} C is called the generalized Fourier transform of f if there exists an integer m N 0 such that f (x)ˆγ(x)dx = ˆf (x)γ(x)dx R s R s is satisfied for all γ S 2m. The smallest such integer m is called the order of ˆf. fasshauer@iit.edu MATH 590 Chapter 7 17
CPD Functions and Generalized Fourier Transforms The Schwartz Space and the Generalized Fourier Transform The generalized Fourier transform is now given by Definition Let f V be complex-valued. A continuous function ˆf : R s \ {0} C is called the generalized Fourier transform of f if there exists an integer m N 0 such that f (x)ˆγ(x)dx = ˆf (x)γ(x)dx R s R s is satisfied for all γ S 2m. The smallest such integer m is called the order of ˆf. Remark Various definitions of the generalized Fourier transform exist in the literature (see, e.g., [Gel fand and Vilenkin (1964)]). fasshauer@iit.edu MATH 590 Chapter 7 17
CPD Functions and Generalized Fourier Transforms The Schwartz Space and the Generalized Fourier Transform Since one can show that the generalized Fourier transform of an s-variate polynomial of degree at most 2m is zero, it follows that the inverse generalized Fourier transform is only unique up to addition of such a polynomial. fasshauer@iit.edu MATH 590 Chapter 7 18
CPD Functions and Generalized Fourier Transforms The Schwartz Space and the Generalized Fourier Transform Since one can show that the generalized Fourier transform of an s-variate polynomial of degree at most 2m is zero, it follows that the inverse generalized Fourier transform is only unique up to addition of such a polynomial. The order of the generalized Fourier transform is nothing but the order of the singularity at the origin of the generalized Fourier transform. fasshauer@iit.edu MATH 590 Chapter 7 18
CPD Functions and Generalized Fourier Transforms The Schwartz Space and the Generalized Fourier Transform Since one can show that the generalized Fourier transform of an s-variate polynomial of degree at most 2m is zero, it follows that the inverse generalized Fourier transform is only unique up to addition of such a polynomial. The order of the generalized Fourier transform is nothing but the order of the singularity at the origin of the generalized Fourier transform. For functions in L 1 (R s ) the generalized Fourier transform coincides with the classical Fourier transform. fasshauer@iit.edu MATH 590 Chapter 7 18
CPD Functions and Generalized Fourier Transforms The Schwartz Space and the Generalized Fourier Transform Since one can show that the generalized Fourier transform of an s-variate polynomial of degree at most 2m is zero, it follows that the inverse generalized Fourier transform is only unique up to addition of such a polynomial. The order of the generalized Fourier transform is nothing but the order of the singularity at the origin of the generalized Fourier transform. For functions in L 1 (R s ) the generalized Fourier transform coincides with the classical Fourier transform. For functions in L 2 (R s ) it coincides with the distributional Fourier transform. fasshauer@iit.edu MATH 590 Chapter 7 18
CPD Functions and Generalized Fourier Transforms Fourier Transform Characterization This general approach originated in the manuscript [Madych and Nelson (1983)]. Many more details can be found in the original literature as well as in [Wendland (2005a)]. The following result is due to [Iske (1994)]. Theorem Suppose the complex-valued function Φ V possesses a generalized Fourier transform ˆΦ of order m which is continuous on R s \ {0}. Then Φ is strictly conditionally positive definite of order m if and only if ˆΦ is non-negative and non-vanishing. fasshauer@iit.edu MATH 590 Chapter 7 19
CPD Functions and Generalized Fourier Transforms Fourier Transform Characterization This general approach originated in the manuscript [Madych and Nelson (1983)]. Many more details can be found in the original literature as well as in [Wendland (2005a)]. The following result is due to [Iske (1994)]. Theorem Suppose the complex-valued function Φ V possesses a generalized Fourier transform ˆΦ of order m which is continuous on R s \ {0}. Then Φ is strictly conditionally positive definite of order m if and only if ˆΦ is non-negative and non-vanishing. Remark This theorem states that strictly conditionally positive definite functions on R s are characterized by the order of the singularity of their generalized Fourier transform at the origin, provided that this generalized Fourier transform is non-negative and non-zero. fasshauer@iit.edu MATH 590 Chapter 7 19
CPD Functions and Generalized Fourier Transforms Fourier Transform Characterization Since integral characterizations similar to our earlier theorems of Schoenberg for positive definite radial functions are so complicated in the conditionally positive definite case we do not pursue the concept of a conditionally positive definite radial function here. Such theorems can be found in [Guo et al. (1993a)]. fasshauer@iit.edu MATH 590 Chapter 7 20
CPD Functions and Generalized Fourier Transforms Fourier Transform Characterization Since integral characterizations similar to our earlier theorems of Schoenberg for positive definite radial functions are so complicated in the conditionally positive definite case we do not pursue the concept of a conditionally positive definite radial function here. Such theorems can be found in [Guo et al. (1993a)]. Examples of radial functions via the Fourier transform approach are given in the next chapter. In Chapter 9 we will explore the connection between completely and multiply monotone functions and conditionally positive definite radial functions. fasshauer@iit.edu MATH 590 Chapter 7 20
References I Appendix References Buhmann, M. D. (2003). Radial Basis Functions: Theory and Implementations. Cambridge University Press. Fasshauer, G. E. (2007). Meshfree Approximation Methods with MATLAB. World Scientific Publishers. Gel fand, I. M. and Vilenkin, N. Ya. (1964). Generalized Functions Vol. 4. Academic Press (New York). Iske, A. (2004). Multiresolution Methods in Scattered Data Modelling. Lecture Notes in Computational Science and Engineering 37, Springer Verlag (Berlin). Wendland, H. (2005a). Scattered Data Approximation. Cambridge University Press (Cambridge). fasshauer@iit.edu MATH 590 Chapter 7 21
References II Appendix References Guo, K., Hu, S. and Sun, X. (1993a). Conditionally positive definite functions and Laplace-Stieltjes integrals. J. Approx. Theory 74, pp. 249 265. Iske, A. (1994). Charakterisierung bedingt positiv definiter Funktionen für multivariate Interpolationsmethoden mit radial Basisfunktionen. Ph.D. Dissertation, Universität Göttingen. Madych, W. R. and Nelson, S. A. (1983). Multivariate interpolation: a variational theory. manuscript. Sun, X. (1993b). Conditionally positive definite functions and their application to multivariate interpolation. J. Approx. Theory 74, pp. 159 180. fasshauer@iit.edu MATH 590 Chapter 7 22