Physically Based Modeling Implicit Methods for Differential Equations

Size: px

Start display at page:

Download "Physically Based Modeling Implicit Methods for Differential Equations"

Cleopatra Harvey
7 years ago
Views:

1 Pysically Based Modeling Implicit Metods for Differential Equations David Baraff Pixar Animation Studios Please note: Tis document is 21 by David Baraff. Tis capter may be freely duplicated and distributed so long as no consideration is received in return, and tis copyrigt notice remains intact.

2 Implicit Metods for Differential Equations David Baraff Pixar Animation Studios 1 Implicit Metods Te metods we ave looked at for solving differential equations in te first section of tese notes Euler s metod, te midpoint metod are all called explicit metods for solving ODE s. However, sometimes an ODE can become stiff, in wic case explicit metods don t do a very good job of solving tem. Wenever possible, it is desirable to cange your problem formulation so tat you don t ave to solve a stiff ODE. Sometimes owever tat s not possible, and you ave just ave to be able to solve stiff ODE s. If tat s te case, you ll usually ave to use an ODE solution metod wic is implicit. 2 Example Stiff ODE First, wat is te meaning and cause of stiff equations? Lets consider an example tat arises frequently in dynamics. Suppose tat we ave a particle, wit position xt, yt, and suppose tat we want te y-coordinate to always be zero. One way of doing tis is to add a component kyt to ẏt were k is a large positive constant. If k is large enoug, ten te particle will never move too far away from yt, since te kyt term always brings yt back towards zero. However, lets assume tat tere is no restriction on te x-coordinate, and tat we want a user to be able to pull te particle arbitrarily along te x-axis. So lets assume tat over some time interval our differential equation is simply Ẋt d dt xt yt xt kyt. 2 1 We ll also assume tat te particle doesn t start exactly wit y. Wat s appening ere is tat te particle is strongly attracted to te line y, and less strongly towards x. If we solve te ODE far enoug forward in time, we expect te particle s location to converge towards, and ten stay tere once it arrives. Now suppose we use Euler s metod to solve te equation. If we take a step of size,weget x x X new X + Ẋt +. y ky Tis yields x x X new y ky 1 x 1 ky. D1

3 If we look at te y component of tis equation, we see tat if 1 k > 1 ten te y new we compute will ave an absolute value wic is larger tan y. In oter words, if 1 k > 1, Euler s metod will not converge to an answer: eac step will result in a value of y new wic is larger tan te last. Tecnically, Euler s metod is unstable for 1 k > 1. Tus, we better ave 1 k > 1ork < 2 if we ope to converge. Te largest step we can ope to take is less tan 2/k. Now, if k is a large number, we ll ave to take very small steps. Tis means tat te particle slides towards, excruciatingly slowly. Even toug te particle may nearly satisfy y, we ave to take suc small steps tat te particles progress along te x-axis is pretty muc nonexistent. Tat s te embodiment of a stiff ODE. In tis case, te stiffness arises from making k very large in order to keep te particle close to te line y. Later on, wen we connect particles wit secondorder dynamics togeter wit springs, we ll experience exactly te same effect: stiff ODE s. Even if we use a more sopisticated explicit metod suc as fourt-order Runge-Kutta, we may do a little better in te size of our steps, but we ll still ave major problems. Now as we said above, te name of te game is to pose your dynamics problems so tat you don t experience stiff ODE s. However, wen you can t, you ll ave to turn towards an implicit solution metod. Te metod we ll sow below is te simplest of te implicit metods, and its based on taking an Euler step backwards. 3 Solving Stiff ODE s Given a differential equation d Xt f Xt, dt te explicit Euler update would be X new X + f Xt, to advance te system forward in time. For a stiff problem toug, we cange te update to instead be X new X + f X new 3 1 Tat is, we re going to evaluate f at te point we re aiming at, rater tan were we came from. If you tink about reversing te world and running everyting backwards, te above equation makes perfect sense. Ten te equation says if you were at X new, and took a step f X new, you d end up at X. So if your differential equation represents a system tat is reversible in time, tis step makes sense. It s just finding a point X new suc tat if you ran time backwards, you d end up at X. So, we re looking for a X new suc tat f, evaluated tere, times, points directly back at were we came from. Unfortunately, we can t in general solve for X new, unless f appens to be a linear function. To cope wit tis, we ll replace f X new wit a linear approximation, again based on f s Taylor series. Lets define X by X X new X. Using tis, we rewrite equation 3 1 as or just Next, lets approximate f X + X by X + X X + f X + X. X f X + X. f X + f X X. SIGGRAPH 21 COURSE NOTES D2 PHYSICALLY BASED MODELING

4 Note tat since f X is a vector, te derivative f X is a matrix. Using tis approximation, we can approximate X wit or X f X + f X X. X f X X f X Rewriting tis as 1 I f X X f X, were I is te identity matrix, we can solve for X as X 1 1 I f X f X 3 2 Computing X new X + X is clearly more work tan using an explicit metod, since we ave to solve a linear system at eac step. Wile tis would seem a serious weakness, computationally speaking, don t despair yet. For many types of problems, te matrix f will be sparse for example, if we are simulating a spring-lattice, f will ave a structure wic matces te connectivity of te particles. For a discussion of sparsity and solution tecniques, see Baraff and Witkin [1]. Basic material in Press et al. [2] will also prove useful. As a result, it is usually possible to solve equation 3 2 in linear time i.e. time proportional to te dimension of X. In suc cases, te payoff is dramatic: we can usually take considerably large timesteps witout losing stability i.e. witout divergence, as appens wit te explicit case if te stepsize is too large. Te time taken in solving eac linear system is tus more tan offset by te fact tat our timesteps are often orders of magnitude bigger tan we could manage using an explicit metod. Of course, te code needed to do all tis is muc more complicated tan in te explicit case; like we said, make your problems un-stiff if you can, and if not, pay te price. Lets apply te implicit metod to equation 2 1. We ave tat f Xt is Differentiating wit respect to X yields f Xt xt kyt f Xt X f Xt. 1 k Ten te matrix 1 I f X is k 1+ 1+k SIGGRAPH 21 COURSE NOTES D3 PHYSICALLY BASED MODELING

5 Inverting tis matrix, and multiplying by f X yields X 1+ 1+k +1 1+k + 1 x 1 + k ky 1 x ky x ky Wat is te limit on te stepsize in tis case? Te answer is: tere is no limit! In tis case, if we let grow to infinity, we get lim X lim + 1 x x k ky k ky x. y Tis means tat we acieve X new X + X in a single step! For a general stiff ODE, we won t be able to take steps of arbitrary size, but we will be able to take muc larger steps using an implicit metod tan using an explicit metod. Te extra cost of solving a linear equation is more tan made up by te time saved by taking large timesteps. 4 Solving Second-Order Equations Most dynamics problems are written in terms of a second-order differential equation: ẍt f xt, ẋt. 4 1 Tis equation is easily converted to a first-order differential equation by adding new variables. If we define v ẋ, ten we can rewrite equation 4 1 as d xt vt 4 2 dt vt f xt, vt wic is a first-order system. However, applying te backward Euler metod to equation 4 2 results in a linear system of size 2n 2n were n is te dimension of x. A fairly simple transformation allows us to reduce te size of te problem to solving an n n linear system instead. It is important to note tat bot te 2n 2n and n n systems will ave te same degree of sparsity, so solving te smaller system will be faster. Te n n system tat needs to be solved is derived as follows. Let us simplify notation by writing x xt and v vt.wealsodefine x xt + xt and v vt + vt.te backward Euler update, applied to equation 4 2, yields x v v + v f x + x, v + v. 4 3 SIGGRAPH 21 COURSE NOTES D4 PHYSICALLY BASED MODELING

6 Applying a Taylor series expansion to f wic in tis context is a function of bot x and v yields te first-order approximation f x + x, v + v f + f f x + x v v. In tis equation, te derivative f / x is evaluated for te state x, v and similarly for f / v. Substituting tis approximation into equation 4 3 yields te linear system x v v + v f + f f x + x v v. 4 4 Taking te bottom row of equation 4 4 and substituting x v + v yields v f + f x v + v + f v v. Letting I denote te identity matrix, and regrouping, we obtain I f f 2 v f + f v x x v 4 5 wic we ten solve for v. Given v, we trivially compute x v + v. Te above assumes tat te function f as no direct dependence on time; in te case tat f varies directly wit time for example, if f describes time-varying external forces, or references moving points or coordinate frames tat are not variables of x ten equation 4 5 needs an additional term to account for tis dependence: I f f 2 v f + f v x x v + f 4 6 t References [1] D. Baraff and A. Witkin. Large steps in clot simulation. Computer Grapics Proc. SIG- GRAPH, [2] W.H. Press, B.P. Flannery, S.A. Teukolsky, and W.T. Vetterling. Numerical Recipes. Cambridge University Press, SIGGRAPH 21 COURSE NOTES D5 PHYSICALLY BASED MODELING

Derivatives Math 120 Calculus I D Joyce, Fall 2013

Derivatives Math 120 Calculus I D Joyce, Fall 2013 Derivatives Mat 20 Calculus I D Joyce, Fall 203 Since we ave a good understanding of its, we can develop derivatives very quickly. Recall tat we defined te derivative f x of a function f at x to be te