3 Polynomial Interpolation

Transcription

1 3 Polynomial Interpolation Read sections 7., 7., (up to p. 39), Review questions , , , 7.7, 7.8. All methods for solving ordinary differential equations which we considered before were socalled one-step methods. This means that the new approximation y k of the solution y t k depends only on the approximation y k at t k (and the differential equation, the step size, and t k, of course). In order to obtain higher order methods, it was necessary to evaluate the right hand side f many times in each step. If it is expensive to evaluate f, this procedure can be rather timeconsuming. Hence, one might wonder if it would be possible to use previous approximations y k y k m and their function values f t j y j. Such methods are called multi-step methods. In order to describe such methods we need to introduce polynomial interpolation. 3. The Interpolation Problem The general interpolation problem can be described as follows: Let t i y i i n with t t t n be given (discrete) points. Vi assume that the values y i are a function of t i. We are looking for a function y f t such that f t i y i i n Such a function f is called an interpolating function or simply an interpolant. A table with discrete points can be obtained in different applications: The points can be measured values where one wants to find a smooth curve in order to plot the function in a nice way. The points are given rows in a table. One needs function values between the rows of that table. One wants to replace a complex mathematical function by a simpler one which is close to the original one but easier to handle. The table is obtained by sampling the complex function. We have already seen an example of interpolation: When we solved a differential equation numerically, we got precisely such a set of discrete points. The easiest thing to do is to assume that the function behaves linearly between two successive approximations t k y k and t k y k. We may know for sure that the exact solution to the differential equation is not a linear function, but we hope that the linear function lies close to the local exact solution. This assumption can be more or less justified depending on the discrete values and the solution at hand. In Figure we

2 4 Different interpolations of raw data 3 y y Figure : Interpolation of a numerical solution compare the piecewise linear interpolation of our numerical solution to the van der Pol equation with the exact solution. The piecewise linear interpolation is built-in into MATLAB s plot command. I received the exact solution by using a sophisticated interpolation algorithm which is included with ode Polynomial Interpolation by Monomials As we have seen before, piecewise linear interpolation is a relatively bad way of interpolating a function. How can one construct better interpolating functions? Taking into account the computational expense one should choose the functions as simple as possible. If we do not have other informations about the function available, the best what we can do is probably to use polynomials. They are very easy to handle and hopefully flexible enough. The polynomial interpolation problems reads: Find a polynomial p n t with a degree of at most n such that p n t i y i i n A polynomial can be written down as p n t x t x 3 t x n t n where x n are the coefficients. Such a representation of p n is called a monomial basis representation. In the above ansatz, the coefficients are the unknowns. More precisely, x n must be determined such that the interpolating conditions p t i y i are fulfilled, that is, x t i x 3 t i x n t n i y i i n

3 These equations can be written down as a linear system of equations in the unknown coefficients: t t t n t t t n t n t n t n n x. x n y y. y n A x y This linear system of equations can be solved uniquely for the coefficients x n if the matrix A is nonsingular. One can show that the matrix A is nonsingular if all t i are pairwise different. The matrix A is called a Vandermonde matrix. Example 3.. Let our given points be (-,-7), (0,-), and (,0). We want to find a second degree polynomial which interpolates these points. The linear system to be solved is t t x t t y x y t 3 t3 x 3 y 3 where t, t 0, t 3, and y 7, y, y 3 0. We obtain the system 4 x x x 3 0 The solution is, x 5, x 3 4. The polynomial is p t 5t 4t For completeness, I include a short MATLAB program which realizes monomial interpolation. For reasons that will be clear later avoid to use it in practice! As usual, all vectors are assumed to be column vectors. function x = PolyM(ti,yi) n = length(ti); A = ones(n); for i = :n A(:,i) = A(:,i-).*ti; end x = A yi; The interpolation polynomial by itself is not very interesting. What one wants to do is to evaluate the polynomial for a set of values for the independent variable t. If one does this in the naive way, the amount of work may be rather large. For a given t and set of coefficients x i, one 3

4 needs i multiplications in order to form the term x i t i. The overall computational cost is the sum of all these multiplications and, additionally, n additions: n i i n n n n O n This is too expensive! Fortunately, there is a better algorithm called Horner s algorithm. The idea is to introduce parentheses in the following way (here demonstrated for n 6): p 5 t x 6 t 5 x 5 t 4 x 4 t 3 x 3 t x t x 6 t 4 x 5 t 3 x 4 t x 3 t x t x 6 t 3 x 5 t x 4 t x 3 t x t x 6 t x 5 t x 4 t x 3 t x t The number of operations is now proportional to the degree n such that the computation is much faster. The following MATLAB function realizes Horner s algorithm. function pt = HornerM(t,x) n = length(x); pt = x(n); for i = n-:-: pt = pt*t+x(i); end What are the most important properties of monomial interpolation? Advantages of monomial interpolation The polynomial is easy to write down. There is an easy algorithm for the computation of the system matrix A. The polynomial can be easily and efficiently evaluated. Disadvantages of monomial interpolation It is very expensive to solve the linear system Ax b. This amounts to order O n 3 operations. The linear system is very often ill-conditioned. This means that the algorithm is numerically unstable. An example will be given later. 4

5 3.3 Lagrange Polynomial In order to avoid the drawbacks with the monomial representation, one can try to write down the interpolating polynomial in such a way that the linear system of equations is much better conditioned and much easier to solve. The simplest ansatz in this respect would be if the matrix A in the linear system were the unit matrix. In that case, we would simply obtain x i y i such that there are no computations involved. Such a representation exists. It is called Lagrange interpolation. The interpolation polynomial is written down as p n t y l t y n l n t The polynomials l j t are chosen in such a way that i l j t i i j 0 j If this property holds true, obviously p n t i y i. The polynomials l j t can be explicitly written down: l j t k k n k j n k j t t k t j t k Example 3.. We will take the same data as before: (-,-7), (0,-), (,0). The Lagrange polynomial is t t t t 3 t t p t y t t 3 t t y t t y 3 t t t t 3 t t t t 3 t 3 t t 3 t For our particular data, this becomes t t t t p t 7 9 t t t t Since the interpolating polynomial is unique, this is just the same polynomial as in the previous example written down in a different way. Advantages of Lagrange interpolation The interpolation polynomial can be written down without the solution of a linear system of equations. The basis polynomials l j t depend only on t t n and not on the values y i. Disadvantages of Lagrange interpolation It is extremely expensive to evaluate the polynomial. The Lagrange polynomial is only used for theoretical considerations. 5

6 3.4 Newton Polynomial There is a third possibility of writing down the interpolating polynomial which combines numerical stability with efficient algorithms. This is Newton interpolation. The ansatz for the polynomial reads p n t x t t x 3 t t t t x n t t t t t t n Let us now write down the linear system of equations Ax b which one obtains if one introduces the interpolation conditions p n t i y i. The resulting matrix A is triangular! A t t 0 0 t 3 t t 3 t t 3 t 0 t n t t n t t n t t n t t n t t n t n Hence, the system is easy to solve and the number of operations is O n. Moreover, the condition number is moderate. Example 3.3. We take the same data as before, (-,-7), (0,-), (,0). The linear system becomes 0 0 x 0 7 x 3 3 x 3 0 whose solution is easy to obtain: 7, x 3, x 3 4. This leads to the interpolant p t 7 3 t 4 t t If one combines the terms with equal powers of t, the result is p t 5t 4t. This is identical to the monomial representation as we expect because of the uniqueness of the interpolating polynomial. An algorithm for computing the Newton polynomial could look like this: function x = PolyN(ti,yi) n = length(ti); A = zeros(n); A(:,) = ones(n,); for i = :n A(i:end,i) = A(i:end,i-).*(ti(i:end)-ti(i-)); end x = A yi; 6

7 This algorithm uses the fact that MATLAB s backslash operator sees that A is a triangular matrix and uses this information. Implementations in other programming languages which are not vector-oriented use a different algorithm basing on divided differences which makes the computation even more efficient. The computation of a polynomial value for a given argument can be done by a modified Horner s algorithm. The expression used can be derived as follows: p 5 t x 6 t t 5 t t 4 t t 3 t t t t x 5 t t 4 t t 3 t t t t x 4 t t 3 t t t t x 3 t t t t x t t x 6 t t 5 t t 4 t t 3 t t x 5 t t 4 t t 3 t t x 4 t t 3 t t x 3 t t x t t x 6 t t 5 t t 4 t t 3 x 5 t t 4 t t 3 x 4 t t 3 x 3 t t x t t x 6 t t 5 x 5 t t 4 x 4 t t 3 x 3 t t x t t A MATLAB implementation could look like this: 7

8 3.5 Interpolating a Function function pt = HornerN(t,x,ti) n = length(ti); pt = x(n); for i = n-:-: pt = pt*(t-ti(i))+x(i); end Here we are shifting our point of view a little bit. Instead of taking the data points t i y i as given somehow, we assume that we want to interpolate a (maybe complex) function by polynomials such that the interpolant is a good approximation to our function. Let f t be the given function. The data points are now t i f t i sampled for n selected arguments. One could ask the question if a growing number of sample points leads to a better approximation to our function. The following example shows that this is not true in general. Example 3.4. Runge s Example. The function we are interested in approximating by polynomials is f t 5t where t We choose n equidistant points in, that is, h n; t i i h i 0 n Figure contains the exact function as well as interpolation results for n 5 0. Runge s Example function 0th order 5th order Figure : Runge s example Such large oscillations around the interval boundaries are called Runge s phenomenon. These oscillations are typical for interpolating polynomials of higher degree and equidistant interpolation nodes. They can only be avoided if special interpolation nodes (known as Chebyshev points) 8

9 are chosen. In that case, even convergence can be assured. But there is another possibility for avoiding a too high degree of an interpolating polynomial. For that aim, one can apply piecewise polynomials of lower degree. This will be the topic of later lectures. 3.6 Symptoms of Ill-Conditioning We will describe shortly how ill-conditioning can effect the results of polynomial interpolation. The following subsection is motivated by Gerd Eriksson, Numeriska algoritmer med MATLAB. We will study how the interpolation algorithms work with the following input data: t y We will compare two different algorithms for computing the 5th-order polynomial which interpolates the given data. The first algorithm is Newton interpolation. It is easy to see that we obtain a good approximation. The second algorithm is the monomial representation. In Figure 3, we plotted the two curves which are the result of the interpolation algorithms. Note that, in theory, they should be identical Two Interpolation Algorithms Monomial Newton Figure 3: A bad interpolation algorithm For completeness, I will include the algorithm which I used to obtain the plot: 9

10 clear npl = 00; ti = [00.5;0.5;0.5;03;04;05]; yi = [3;.5;.5;;;0]; xn = PolyN(ti,yi); xm = PolyM(ti,yi); L = ti(end)-ti(); h = L/npl; tpl = (0:npl) *h+ti(); fn = zeros(size(tpl)); fm = zeros(size(tpl)); for i = :length(tpl) fn(i) = HornerN(tpl(i),xn,ti); fm(i) = HornerM(tpl(i),xm); end plot(tpl,fm,tpl,fn, -- ) title( Two Interpolation Algorithms ) legend( Monomial, Newton,0) We will investigate the reason for this unexpected result. There are three main algorithmic steps involved: Computation of the system matrix A; solution of the linear system Ax y; computation of the polynomial values. The columns of the Vandermonde matrix consist of successively increasing potentials of the arguments. The magnitude of the elements increases from one in the first column to around 0 5 in the last column. This matrix becomes very ill-conditioned. An explicit computation of the condition number gives The next step consists of the solution of the linear system. The accuracy of the computer is not sufficient to provide a reliable answer. MATLAB gives a warning during program execution this should be taken seriously! Taking this condition number literally, we would not expect any valid digit in the final result. The figure above is a little bit more optimistic. In fact, one can see experimentally that, in practice, one correct digit can be expected. We give the coefficients with three digits even if they are not reliable. It is only to show the huge difference in magnitude between them x 35 0 x x x x The insecurity in the coefficients is not the reason for the jagged behavior of the plot. Even if the coefficients are perturbed, a fifth degree polynomial is a smooth curve. It cannot have more than two maxima and two minima. Therefore, the computation by Horner s algorithm must be prone 0

11 to errors. In order to show what is happening, let us compute the function value p The computed polynomial value amounts to e 00 which is in error by 44%. The intermediate results in algorithm HornerM are: e e e e e e+00 The intermediate results are of a rather huge magnitude and have different sign. It is only in the last step where a result of the magnitude one appears. Compared to the result before which has magnitude 0 this indicates a catastrophic cancellation of terms. It appears even in the previous steps although is this not seen explicitly.