3. Interpolation. Closing the Gaps of Discretization... Beyond Polynomials

Transcription

1 3. Interpolation Closing the Gaps of Discretization... Beyond Polynomials Closing the Gaps of Discretization... Beyond Polynomials, December 19,

2 3.3. Polynomial Splines Idea of Polynomial Splines Remember the main disadvantages of polynomial interpolation: Number of data points and polynomial degree are functionally related. For bigger n (and therefore for a bigger number of nodes, which frequently occurs in practice), polynomial interpolation is useless just because of matters of condition. Idea for remedy: Do something similar as ship builders did to define harmonic curves with the help of an elastic wooden slat (spline) fixed at an arbitrary number of support points. The resulting shape minimizes energy. wikipedia Closing the Gaps of Discretization... Beyond Polynomials, December 19,

3 Definition of Polynomial Splines Glue together pieces of polynomials of lower degree to form a global interpolant for a big number of nodes as well. This is exactly what polynomial splines or, in short, splines do, which we introduce in a general way, without specific references to interpolation problems. Let again be a = x 0 < x 1 <... < x n = b and m N. The x i are called knots. We only examine the special case of simple knots i.e. x i x j for i j. A function s : [a, b] R is called spline of order m or of degree m 1, if the following holds: s(x) = p i (x) on [x i, x i+1 ] with p i P m 1, i = 0, 1,..., n 1, s C m 2 ([a, b]). Between two neighboring knots s is a polynomial of degree m 1 and globally (particularly in the knots themselves) s is m 2 times continuously differentiable. s is also called piecewise polynomial. Closing the Gaps of Discretization... Beyond Polynomials, December 19,

4 Examples, Characteristics, and Applications What does a polynomial spline look like? For this, we examine the simplest cases: m = 1: s is a step function (piecewise continuous, continuity is not even given in knots) m = 2: s is a polygonal chain (piecewise linear, continuous in knots) m = 3: quadratic spline (piecewise quadratic, globally one time continuously differentiable) m = 4: cubic spline (piecewise cubic, globally twice continuously differentiable) Evidently, with fixed knots and fixed degree, the set of all splines forms a vector space with dimension n + m 1: a polygon p 0 P m 1 has m degrees of freedom our slpine consists of n such polynomials the spline is m 2 times continuously differentiable, this results in m 1 conditions per inner node we have n 1 inner nodes m n (m 1) (n 1) = n + m 1. Closing the Gaps of Discretization... Beyond Polynomials, December 19,

5 Examples, Characteristics, and Applications fourth order interpolating polynomial for five support points linear spline (m = 2) euclid.colorado.edu Closing the Gaps of Discretization... Beyond Polynomials, December 19,

6 Examples, Characteristics, and Applications fourth order interpolating polynomial for five support points quadratic spline (m = 3) euclid.colorado.edu Closing the Gaps of Discretization... Beyond Polynomials, December 19,

7 Examples, Characteristics, and Applications fourth order interpolating polynomial for five support points quadratic spline (m = 3) euclid.colorado.edu Closing the Gaps of Discretization... Beyond Polynomials, December 19,

8 Examples, Characteristics, and Applications fourth order interpolating polynomial for five support points quadratic spline (m = 3) euclid.colorado.edu Closing the Gaps of Discretization... Beyond Polynomials, December 19,

9 Examples, Characteristics, and Applications fourth order interpolating polynomial for five support points natural cubic spline (m = 4, zero second derivative at end points) euclid.colorado.edu Closing the Gaps of Discretization... Beyond Polynomials, December 19,

10 Examples, Characteristics, and Applications fourth order interpolating polynomial for five support points clamped cubic spline (m = 4, zero first derivative at end points) euclid.colorado.edu Closing the Gaps of Discretization... Beyond Polynomials, December 19,

11 Main fields of application for splines: Interpolation: A spline is to be constructed in a way such that it fulfills n + m 1 interpolation constraints according to the number of its degrees of freedom. The nodes of the interpolation can but do not have to be knots. CAGD: Here the spline is to be constructed in a way that it takes the shape of the curve defined by control points (important in context with free form curves and surfaces). Closing the Gaps of Discretization... Beyond Polynomials, December 19,

12 Interpolation with Cubic Splines Now we actually use splines for interpolation. Cubic splines, the case of m = 4, are widespread for interpolation purposes. Moreover, they are easy to construct and to analyze. Examine the following special case: simpe knots: a = x 0 < x 1 <... < x n = b locally: spline s P 3 on every subinterval [x i, x i+1 ], i = 0, 1,..., n 1 globally: spline s C 2 ([a, b]) Let the nodes be exactly the knots, then the interpolation constraints are With this, we get: s(x i ) = y i, i = 0, 1,..., n. We have n + m 1 = n + 3 degrees of freedom for the determination of the concrete spline (dimension of the underlying vector space, see above) Of those, n + 1 are needed for the above interpolation constraints. With this, 2 degrees of freedom remain, which we can use for further conditions to s. More on this later. Firstly, we set about building our cubic spline. Closing the Gaps of Discretization... Beyond Polynomials, December 19,

13 The Local Polynomial Segments For every subinterval [x i, x i+1 ] compute the local cubic polynomial: s(x) = p i (x) = at 3 + bt 2 + ct + d with t = x x i x i+1 x i Using the function values y i and y i+1 at the support points and given derivatives y i and y i+1 d = y i c = (x i+1 x i )y i, a + b + c + d = y i+1 3a + 2b + c = (x i+1 x i )y i+1 This gives the solution d = y i, c = (x i+1 x i )y i, b = 3(y i+1 y i ) 2h i y i h i y i+1, a = 2(y i+1 y i ) + h i y i + h i y i+1 with h i = x i+1 x i and, finally, the cubic polynomial p i (x) = (1 3t 2 + 2t 3 )y i + (3t 2 2t 3 )y i+1 + (t 2t 2 + t 3 )h i y i + ( t 2 + t 3 )h i y i+1. Closing the Gaps of Discretization... Beyond Polynomials, December 19,

14 The Global Gluing Together In the representation using local polynomials p i (x) = (1 3t 2 + 2t 3 )y i + (3t 2 2t 3 )y i+1 + (t 2t 2 + t 3 )h i y i + ( t 2 + t 3 )h i y i+1, the knot values y 0,..., y n are given by the constraints of interpolation. The continuity of the spline and the continuity of the first derivative are already included in our representation. Therefore, we have n + 1 degrees of freedom (y 0,..., y n ) left to ensure continuity of the second derivative at the n 1 interior knots. This means, we need two additional conditions to get a unique solution. We differentiate all local polynomials twice and obtain continuity at the n 1 glued points by equating the second derivative of p i 1 in t = 1 and the second derivative of p i in t = 0: ( y i y i + 2 h i 1 h i 1 h i ) + y ( 1 yi y i 1 i+1 = 3 h i hi y i+1 y i h 2 i ). Closing the Gaps of Discretization... Beyond Polynomials, December 19,

15 Obviously, the equations for the continuity of the second derivative establish a tridiagonal system of linear equations h 0 h 0 h 1 h h n 2 h n 2 h n 1 h n 1 y 0... y n = ( ) y of dimension (n 1) (n + 1) with rhs i = 3 i y i 1 + y i+1 y i a first h i 1 2 h i 2 motivation to intensely deal with the numerical solution of systems of linear equations in chapters 5 and 6. rhs 1.. rhs n 1 Closing the Gaps of Discretization... Beyond Polynomials, December 19,

16 How should the conditions for the two spare degrees of freedom be defined? Usually, boundary conditions are being set. In this case, three methods are common: providing the slope at the boundaries, i.e. of y 0 and y n providing the second derivatives at the boundaries (natural boundary conditions) periodic boundary conditions: The first and second derivatives at both boundary points respectively have to be equal, thus y 0 = y n and s (x 0 ) = s (x n). For natural boundary conditions, e.g., we get the quadratic system of linear equations h 0 h h 0 h 0 h 1 h h n 2 h n 2 h n 1 h n h n 1 h n 1 y 0... y n = 3 y 1 y 0 h 0 2 rhs 1.. rhs n 1 3 yn y n 1 h 2 n 1. Closing the Gaps of Discretization... Beyond Polynomials, December 19,

17 Costs and Performance How much does cubic spline interpolation cost? A (depending on the choice of boundary conditions possibly slightly perturbed) tridiagonal system of equations has to be solved. It s easy to realize that this can be done with O(n) arithmetic operations (e.g. by Gaussian elimination known from linear algebra). This is to be compared with the O(n 3 ) arithmetic operations (reminiscence to linear algebra or see chapter 4) used for solving a full system of equations at the incidence sample or rather with the quadratic computational effort of the scheme of Aitken and Neville. Which accuracy does the spline operation provide? As we are using local polynomials of third degree and the nodes have distance h := x i+1 x i, the error estimation delivers (analogously to section 3.2) f (x) s(x) = O(h 4 ). This has to be compared with the order of error O(h n+1 ) of polynomial interpolation. However, there, an extremely high differentiability is assumed and, for bigger n, the problems of condition detected earlier occur. Closing the Gaps of Discretization... Beyond Polynomials, December 19,

18 3.4. Trigonometric Interpolation The Principle of Trigonometric Interpolation trigonometric interpolation is used to interpolate periodoc functions f with (after suitable scaling) f (t + 1) = f (t). We use sinus and cosinus functions with different frequencies k N 0 as a basis for the space of our interpolating functions. In the complex plane, these are A representation of a function p as z(t) k with z(t) = e 2πit = cos(2πkt) + i sin(2πkt). p(t) = n c k z(t) k = k=0 n c k (cos(2πkt) + i sin(2πkt)) k=0 is also called representation in frequency space. Closing the Gaps of Discretization... Beyond Polynomials, December 19,

19 As an example consider the function plottet in these graphs: The function can be written as a combination of three frequency modes (k = 1, 3, 6): ( ) p(t) = 4 sin(2πt) + 2 sin(6πt) sin(12πt) = Im 4e 2πit + 2e 6πit e 12πit. Closing the Gaps of Discretization... Beyond Polynomials, December 19,

20 Similar to polynomial interpolation, we use n support points ( t j = j ) n, v j, j = 0,..., n 1. To be found is the interpolant p(z) = n 1 c k z k with z = e 2πit, t [0, 1] k=0 fulfilling the interpolation constraints p(z j ) = v j for j = 0, 1,..., n 1 with z j = e 2πij n. To find this p means de facto to calculate the coefficients c k, those being nothing else but the coefficients of the (discrete) Fourier transform. For this problem, we discuss an almost legendary algorithm, the fast Fourier transform (FFT), in the following. Closing the Gaps of Discretization... Beyond Polynomials, December 19,

21 The Discrete Fourier Transform With the p introduced earlier and the abbreviation ω := e 2πi n the following problem has to be solved: Find n complex numbers c 0,..., c n 1, which fulfill v j = p(ω j ) = n 1 c k ω jk for j = 0, 1,..., n 1. k=0 With a bit of calculus ( ω complex conjugate to ω), it can be shown that c k = 1 n 1 v j ω jk for k = 0, 1,..., n 1. n j=0 We denote by c and v the n-dimensional vectors of the discrete Fourier coefficients or of the DFT input, respectively. Furthermore, let the matrix M be given as M = (ω jk ) 0 j,k n 1. Then, in matrix vector notation, we have v = M c, c = 1 n M v. Closing the Gaps of Discretization... Beyond Polynomials, December 19,

22 The formula for calculating the coefficients is given by the discrete Fourier transform (DFT) of the input data v k. The formula for calculating the values v j out of the Fourier coefficients c k is called inverse discrete Fourier transform (IDFT). Obviously, the number of arithmetic operations needed for DFT and IDFT is of order O(n 2 ). In contrast, the FFT algorithm does with O(n log n) operations for suitable n. Closing the Gaps of Discretization... Beyond Polynomials, December 19,

23 The FFT Algorithm Basic Principle (1) Basic idea: Break down sums of length n into two partial sums of half length and continue recursively until reaching the trivial sum of length 1. For this purpose, we confine ourselves to the case of n = 2 p, i.e. to powers of 2, from the outset. In the following, we examine the inverse discrete Fourier transform (IDFT). The method for the actual DFT follows from this clearly through transitioning to the complex conjugate and dividing by n: v = IDFT (c), c = DFT (v) = 1 IDFT ( v). n Closing the Gaps of Discretization... Beyond Polynomials, December 19,

24 The FFT Algorithm Basic Principle (2) For v = IDFT (c), we get with n = 2m for j = 0, 1,..., m 1 v j = n 1 c k e 2πijk n = k=0 = n/2 1 k=0 m 1 k=0 n/2 1 c 2k e 2πij2k n + k=0 c 2k e 2πijk m 1 m + ω j k=0 c 2k+1 e 2πij(2k+1) n c 2k+1 e 2πijk m. The entries still missing v m,..., v n 1 (the upper half of the indices) emerge, following the same principle, according to v m+j = = for j = 0, 1,..., m 1, as m 1 k=0 m 1 k=0 e 2πi(j+m) n c 2k e 2πi(j+m)2k m 1 n + k=0 c 2k e 2πijk m 1 m ω j = e πi e πij m k=0 c 2k+1 e 2πi(j+m)(2k+1) n c 2k+1 e 2πijk m = e πij m = ω j. Closing the Gaps of Discretization... Beyond Polynomials, December 19,

25 The FFT Algorithm Basic Principle (3) This corresponds to a structure v j = A + ω j B, v m+j = A ω j B, where the components A and B themselves are Fourier sums of half length. We can reduce the computation of v = IDFT (c) to IDFT (c 0, c 2,..., c n 2 ) and IDFT (c 1, c 3,..., c n 1 ), which then have to be linearly combined appropriately to get the first and second half of v. Combining appropriately is to say combining with the aid of the so called Butterfly scheme. Closing the Gaps of Discretization... Beyond Polynomials, December 19,

26 The FFT Algorithm Basic Principle (4) This procedure will now be repeated recursively. With this, we can give a first recursive program in pseudocode: funct IDFT(c[0],...,c[n-1],n); if (n=1) then do v[0]:=c[0] od; else do m:=n/2; z1:=idft(c[0],c[2],...,c[n-2],m); z2:=idft(c[1],c[3],...,c[n-1],m); omega:=exp(2*pi*i/n); for j:=0 to m-1 do v[j]:=z1[j]+omegaˆj*z2[j]; v[m+j]:=z1[j]-omegaˆj*z2[j]; od; return(v[0],...,v[n-1]); od; Closing the Gaps of Discretization... Beyond Polynomials, December 19,

27 Analysis of the FFT Algorithm The basic algorithm above is recursive; for better runtime and memory efficiency, iterative variations were developed. The FFT algorithm naturally breaks down into a sorting phase (rearranging of the c[k]) and a combining phase (butterfly operation). This can be demonstrated for the example of n = 8: Closing the Gaps of Discretization... Beyond Polynomials, December 19,

28 Here, the butterfly operation stands for From this picture, we intuitively learn two things: The underlying principles of the FFT algorithm can be found again in completely different places such as dynamic networks such as Shuffle-Exchange. The computing complexity of the FFT algorithm is of order O(n log(n)). Closing the Gaps of Discretization... Beyond Polynomials, December 19,

29 An Application: Compression of Picture Data Let a gray scale picture be considered as a matrix of gray scale values. Every matrix entry g i,j then corresponds to the gray scale value of the pixel in row i and column j of the picture. To stay independent of the number of the gray scales available, we allow g i,j [0, 1], where zero is equal to the color black. The JPEG method uses the special frequency distribution of natural images to store them at a high compression rate. Losses occurring due to compression are accepted the compression is lossy. The compression is done in the following way: Color images are transformed into the YUV color model (Y stands for intensity, U and V identify hue and saturation). The picture is divided into blocks of 8 8 pixels. In each of those blocks, a fast cosine transform FCT is done which is explained on the next slide. The frequency values are quantized only a certain number of discrete color values is accepted, intermediate values have to be rounded accordingly. This step is mainly responsible for the occurring loss of quality. Finally, the quantized data get compressed appropriately. Closing the Gaps of Discretization... Beyond Polynomials, December 19,

30 The Cosine Transform Image data is neither periodic nor does it have complex parts. Therefore, the conventional Fourier transform is useless here. The JPEG method uses the following variation, the so called Cosine transform: transformation: F k := ( N 1 πk(j f j cos ) ) N j=0 k = 0, 1,..., N 1 inverse transformation: ( f j := 2 N 1 πk(j F k cos ) ) N N k=0 ( := 2 F N 1 0 N 2 + πk(j F k cos ) ) N k=1 j = 0,..., N 1 Instead of the complex exponential terms as basic components, here, only real cosine terms appear. Closing the Gaps of Discretization... Beyond Polynomials, December 19,

31 For 2 D the relevant case for image data processing the following representation results: F k1,k 2 := f j1,j 2 := N 1 1 j 1 =0 4 N 1 N 2 N 2 1 j 2 =0 N 1 1 k 1 =0 ( πk1 (j f j1,j 2 cos ) ) ( πk2 (j cos ) ) N 1 N 2 N 2 1 k 2 =0 ( πk1 (j F k1,k 2 cos ) ) cos N 1 ( πk2 (j ) ) N 2 Closing the Gaps of Discretization... Beyond Polynomials, December 19,

32 Reduction to FFT: We want to use our FFT knowledge in the search for a faster algorithm for solving the discrete cosine transform. For this, we reduce the 2D cosine transform above to the familiar FFT in three steps: The 2D cosine transform can be realized by composing 1D cosine transforms. By mirroring the values of F k or rather f j the cosine transform can be transferred into a modified Fourier transform of double length. With { fj j = 0,..., N 1, fj := and we get fj = 1 2N 2N 1 k=0 F k := F k e ( f 2N j 1 j = N,..., 2N 1 2F k k = 0,..., N 1, 0 k = N, 2F 2N k k = N + 1,..., 2N 1 ) 2πi(j+ 1 2 )k 2N, and Fk = 2N 1 j=0 fj e ( 2πi(j+ 2 1 ) )k 2N. By scaling F k and f j adequately, the introduced standard representation of the (I)DFT results. Closing the Gaps of Discretization... Beyond Polynomials, December 19,

33 3.5. Examples for Applications of Interpolation Interpolation in Computer Science Tables formerly indispensable for functions such as the logarithm or trigonometric functions, today they are actually only used for distribution functions such as the normal distribution print function values at discrete nodes. If intermediate values are wanted, interpolation is necessary. In every function plot discrete points have to be interpolated. Arbitrary nonlinear curves and surfaces so called freeform curves or freeform surfaces play a big role in geometric modeling and computer graphics. For the creation of computer animations, e.g. of cartoon sequences, the explicit creation of every single frame usually is too costly computationally for real-time applications after all, about 25 images per second are needed for a visually appealing (i.e. especially jerk free) animation. For this reason, often only certain keyframes are computed and certain interim frames are interpolated, which is also called inbetweening in this context. When robots are moving (e.g. playing soccer), discrete target points are computed the way from the momentary position to the next target is then determined by interpolation. Closing the Gaps of Discretization... Beyond Polynomials, December 19,

34 For picture reconstruction, the original has to be reconstructed as exactly as possible out of the compressed or faulty image data another case for interpolation. Eventually, the problem of interpolation occurs at every D/A conversion of a signal. Closing the Gaps of Discretization... Beyond Polynomials, December 19,