Iterative Solvers for Linear Systems

Transcription

1 9th SimLab Course on Parallel Numerical Simulation, Iterative Solvers for Linear Systems Bernhard Gatzhammer Chair of Scientific Computing in Computer Science Technische Universität München Germany

2 Outline Poisson-Equation: From the Model to an Equation System Direct Solvers vs. Iterative Solvers Jacobi and Gauss-Seidel Method Residual and Smoothness Multigrid Solver

3 Outline Poisson-Equation: From the Model to an Equation System Direct Solvers vs. Iterative Solvers Jacobi and Gauss-Seidel Method Residual and Smoothness Multigrid Solver

4 The Poisson Equation An Elliptic PDE General: 2 R d, ± 2 R d 1, u :! R u = f in ( = r 2 = r : : : 2 d = f in u = g on ± 1D: = f in 2D: 2 R = f in

5 Why the Poisson Equation is of Importance The Laplace operator is part of many important PDEs: Heat equation: diffusion of heat

6 Why the Poisson Equation is of Importance The Laplace operator is part of many important PDEs: Flow dynamics: friction between molecules

7 Why the Poisson Equation is of Importance The Laplace operator is part of many important PDEs: Structural mechanics: displacement of membrane structures

8 Why the Poisson Equation is of Importance The Laplace operator is part of many important PDEs: Electrics: electric-, magnetic potentials

9 Why the Poisson Equation is of Importance The Laplace operator is part of many important PDEs: Financial mathematics: Diffusive process for option pricing

10 Why the Poisson Equation is of Importance The Laplace operator is part of many important PDEs: Heat equation: diffusion of heat Flow dynamics: friction between molecules Structural mechanics: displacement of membrane structures Electrics: electric-, magnetic potentials Financial mathematics: Diffusive process for option pricing...

11 Discretization with Finite Differences I Find solution to Poisson equation numerically on a computer: Computer has only a finite amount of memory/power discretization Finite Differences as simple discretization Idea: Replace derivative by difference u( + h) u() + h u() u( h) h

12 Discretization with Finite Differences II Use forward and backward differences for 2 nd derivatives: forward diff.: backward 2 ¼ u( + h) u() h u( + h) u() h u( + h) 2u() + u( h) h 2 u( + h h) u( h) h h

13 Grid, Stencil, and Matri 1D Regular Cartesian grid: Nodal equation: Stencil: h i = ih u i 1 2u i + u i+1 = h 2 f i i 1 i i+1 System matri: A = 1 h A 2

14 Grid, Stencil, and Matri 2D Regular Cartesian grid: Nodal equation: h h u i 1;j + u i+1;j + u i;j 1 + u i;j+1 4u i;j = h 2 f i;j i;j = (ih; jh) Stencil: 1 i;j i 1;j i;j 1 i;j 1 i+1;j

15 Grid, Stencil, and Matri 2D System matri: h h A = 1 h B A 1 1 4

16 Discretized Poisson Equation We obtain a system of equations to be solved: Au = f, A 2 R N N, u; f 2 R N N is the number of grid unknowns, i.e. equations f contains sink/source terms and boundary influences We want to obtain u = A 1 f

17 Outline Poisson-Equation: From the Model to an Equation System Direct Solvers vs. Iterative Solvers Jacobi and Gauss-Seidel Method Residual and Smoothness Multigrid Solver

18 Direct Solvers of Systems of Linear Equations Popular direct solvers for Au = f : U = Gaussian elimination LU decomposition Cholesky factorization T N ::: T 2 Au = T N ::: T 2 f, Uu = f 0 A = LU, LUu = f, Uu = L 1 f A = U T U compute u = A 1 f directly up to numerical accuracy computational compleity of GEM in 2D: O(h 6 ), in 3D: O(h 9 ) GEM optimized for band-structure: in 2D: O(h 4 ), in 3D: O(h 7 )

19 Runtime of GEM for Solving Poisson 2D h N runtime (33 TFlop/s) sec sec min 23 sec min 32 sec h 28 min 38 sec d 7 h 28 min 10 sec

20 Runtime of GEM for Solving Poisson 3D h N runtime (33 TFlop/s) min 53 sec h 57 min 26 sec d 26 h 10 min 53 sec a 156 d 57 h 53 min 04 sec

21 Iterative Solvers of Systems of Linear Equations In contrast to direct solvers, iterative solvers Compute approimate solutions u k ¼ A 1 f Improve the approimation in an iterative process One iteration is cheap to compute, in 2D O(h 2 ), in 3D O(h 3 ) Fipoint iteration: Types of methods: u k+1 = F(u k ) k!1! u = F(u) Splitting methods, Krylov-subspace methods, Multigrid methods

22 Outline Poisson-Equation: From the Model to an Equation System Direct Solvers vs. Iterative Solvers Jacobi and Gauss-Seidel Method Residual and Smoothness Multigrid Solver

23 Splitting Methods Split up matri A: Update rule: Au = f (S + T )u = f Su k+1 = f T u k u k+1 = S 1 (f T u k ) u k+1 = F (u k ) Matri S has to be of simple form to to get u k+1

24 The Jacobi Method Choose: Splitting: Update rule: S = diag(a) := D A A = S + T = D A + (A D A ) Au = f (D A + (A D A ))u = f D A u k+1 = (f (A D A )u k ) u k+1 = D 1 A (f (A D A)u k ) u k+1 i = 1 a ii (f i NX a ij u k j ) j=1 j6=i

25 Jacobi Method Eample 1D Heat Equation Stationary heat equation with u as temperature: Boundaries have zero temperature, no 2 2 = f in f = 0 Solution is known: u = 0 u k+1 i = 1 a ii (f i NX j=1 j6=i a ij u j ) =) u k+1 i = 1 2 (uk i 1 + u k i+1)

26 Jacobi Method Eample 1D Heat Equation Unit domain: := [0; 1]; h = 1 8 ; N = A = B A 1 2

27 Jacobi Method Eample 1D Heat Equation Random initial value u 0 = rand(n)

28 Jacobi Method Eample 1D Heat Equation k = 0 0 u k+1 = (uk i 1 + u k i+1) i

29 Jacobi Method Eample 1D Heat Equation k = 1

30 Jacobi Method Eample 1D Heat Equation k = 2

31 Jacobi Method Eample 1D Heat Equation k = 3

32 Jacobi Method Eample 1D Heat Equation k = 4

33 Jacobi Method Eample 1D Heat Equation k = 5

34 Jacobi Method Eample 1D Heat Equation k = 6

35 Jacobi Method Eample 1D Heat Equation k = 7

36 Jacobi Method Eample 1D Heat Equation k = 8

37 Jacobi Method Eample 1D Heat Equation k = 8 Initial values

38 The Gauss-Seidel Method Choose: Splitting: Update rule: S = L(A) := L A A = S + T = L A + (A L A ) Au = f (L A + (A L A ))u = f L A u k+1 = (f (A L A )u k ) u k+1 = L 1 A (f (A L A)u k ) u k+1 i = 1 a ii (f i Xi 1 j=1 a ij u k+1 j NX j=i+1 a ij u k j )

39 Gauss-Seidel Method Eample 1D Heat Equation Stationary heat equation with u as temperature: Boundaries have zero temperature, no 2 2 = f in f = 0 Solution is known: u = 0 u k+1 i = 1 a ii (f i Xi 1 j=1 a ij u k+1 j NX j=i+1 a ij u k j ) =) u k+1 i = 1 2 (uk+1 i 1 + uk i+1)

40 Gauss-Seidel Method Eample 1D Heat Equation Random initial value u 0 = rand(n)

41 Gauss-Seidel Method Eample 1D Heat Equation k = 0

42 Gauss-Seidel Method Eample 1D Heat Equation k = 0

43 Gauss-Seidel Method Eample 1D Heat Equation k = 0

44 Gauss-Seidel Method Eample 1D Heat Equation k = 0

45 Gauss-Seidel Method Eample 1D Heat Equation k = 0

46 Gauss-Seidel Method Eample 1D Heat Equation k = 0

47 Gauss-Seidel Method Eample 1D Heat Equation k = 0

48 Gauss-Seidel Method Eample 1D Heat Equation k = 1

49 Gauss-Seidel Method Eample 1D Heat Equation k = 2

50 Gauss-Seidel Method Eample 1D Heat Equation k = 3

51 Gauss-Seidel Method Eample 1D Heat Equation k = 4

52 Gauss-Seidel Method Eample 1D Heat Equation k = 5

53 Gauss-Seidel Method Eample 1D Heat Equation k = 6

54 Gauss-Seidel Method Eample 1D Heat Equation k = 7

55 Gauss-Seidel Method Eample 1D Heat Equation k = 8

56 Gauss-Seidel Method Eample 1D Heat Equation k = 8 Initial values

57 Gauss-Seidel Method Eample 1D Heat Equation k = 8 Jacobi after 8 iterations

58 Outline Poisson-Equation: From the Model to an Equation System Direct Solvers vs. Iterative Solvers Jacobi and Gauss-Seidel Method Residual and Smoothness Multigrid Solver

59 Residual Residual: Au = f Au k 6= f f Au k = res k Splitting: Au = (S + T )u = f u k+1 = S 1 (f T u k ) u k+1 = S 1 (f (A S)u k ) u k+1 = S 1 (f Au k ) + u k u k+1 = u k + S 1 res k

60 Residual for 1D Heat Equation Eample Local influences form residual: u i 1 2u i + u i+1 = 0 u k i 1 2u k i + u k i+1 = res k i GS k = 0: high residual oscillatory error/fct values GS k = 8: low residual smooth error/fct values

61 Smoothness Observation Smoothnes depends on grid resolution h : Error reduction h = 1 : 8 h = 1 : 4 k! k + 1 h = 1 : 2

62 Outline Poisson-Equation: From the Model to an Equation System Direct Solvers vs. Iterative Solvers Jacobi and Gauss-Seidel Method Residual and Smoothness Multigrid Solver

63 Geometric Multigrid Idea Only high error frequencies are removed effectively Frequencies depend on grid resolution Idea: Use several grid levels Use splitting method to smooth error on each grid level

64 Multigrid Eample 1D Heat Equation Random initial value u 0 = rand(n)

65 Multigrid Eample 1D Heat Equation k = 0, N = 1

66 Multigrid Eample 1D Heat Equation k = 0, N = 1

67 Multigrid Eample 1D Heat Equation k = 0, N = 3

68 Multigrid Eample 1D Heat Equation k = 0, N = 3

69 Multigrid Eample 1D Heat Equation k = 0, N = 7

70 Multigrid Eample 1D Heat Equation k = 1

71 Thank you for your attention!