BigDFT An introduction

Transcription

1 Atomistic and molecular simulations on massively parallel architectures CECAM PARIS, FRANCE An introduction Thierry Deutsch, Damien Caliste, Luigi Genovese L_Sim - CEA Grenoble 17 July 2013

2 Objectives of the lesson To know what are the main parameters in a DFT calculation To run a calculation (i.e. the physicist problem) 1 2 Exchange-correlation 3 High-Performance Computing (parallel and GPU) 4

3 A basis for nanosciences: the project STREP European project: ( ) Four partners, 15 contributors: CEA-INAC Grenoble, U. Basel, U. Louvain-la-Neuve, U. Kiel Aim: To develop an ab-initio DFT code based on Daubechies Wavelets, to be integrated in ABINIT, distributed freely (GNU-GPL license) L. Genovese, A. Neelov, S. Goedecker, T. Deutsch, et al., Daubechies wavelets as a basis set for density functional pseudopotential calculations, J. Chem. Phys. 129, (2008)

4 version 1.7: capabilities Free, surface and periodic boundary conditions Geometry optimizations (with constraints) Born-Oppenheimer Molecular Dynamics Saddle point searches (Nudged-Elastic Band Method) Vibrations External electric fields Unoccupied KS orbitals Collinear and Non-collinear magnetism All functionals of the ABINIT package Hybrid functionals Empirical van der Waals interactions (many flavors) Also available within the ABINIT package

5 Kohn-Sham formalism H-K theorem: E is an unknown functional of the density E = E[ρ] Density Functional Theory Kohn-Sham approach Mapping of an interacting many-electron system into a system with independent particles moving into an effective potential. Find a set of orthonormal orbitals Ψ i (r) that minimizes: E = 1 2 N/2 i=1 N/2 ρ(r) = i=1 Ψ i (r) 2 Ψ i (r)dr + 1 ρ(r)v H (r)dr 2 + E xc [ρ(r)] + V ext (r)ρ(r)dr Ψ i (r)ψ i (r) 2 V H (r) = 4πρ(r)

6 Spirit of Few input files (posinp.xyz, input.dft) Few input parameters ex. not parameters for parallel calculations Use input files with mandatory parameters Optional input files as input.kpt (for periodic systems) or input.perf (parameters for performance)

7 How to run Some pre-processing bigdft-tool nprocs [name] Help to find the optimal number of processors for the run that fit into the computer memory. Also useful to check that all input files are without errors. The main run mpirun -np 8 bigdft [name] tee [name].log Number of MPI tasks : 8 OpenMP parallelization : Yes Maximal OpenMP threads per MPI task : 1 Material acceleration : The outputs No #iproc=0 log is output on stdout, so redirect it. generated data are stored in a subdirectory called data[-name].

8 The input files (or not) posinp.xyz: the atomic coordinates and box definition. psppar.element: pseudo-potentials for each element; input.dft: DFT related parameters; input.geopt: geometrie relaxations and MD; input.kpt: the k-point mesh and band structure path; input.mix: diagonalisation mechanism; input.sic, tddft, occ, perf: other parameters; Only the coordinates are mandatory. All lines of different input files are mandatory and output on log. When is run, all default values are output in files called default.xxx. Naming scheme All input files may be renamed by providing a name argument to the command line i.e. bigdft name.

9 Atomic positions: posinp.xyz or posinp.ascii units reduced, angstroem, bohr or atomic Boundary conditions periodic, surface 8 reduced periodic Si Si Si Si Si Si Si Si

10 Atomic positions: posinp.xyz or posinp.ascii units reduced, angstroem, bohr or atomic Boundary conditions periodic, surface 3 angstroem surface O H H

11 Atomic positions: posinp.xyz or posinp.ascii units reduced, angstroem, bohr or atomic Boundary conditions periodic, surface 5 atomic Si H H H H

12 Atomic coordinate format uses XYZ format plus additional modifications: physical unit, after the number of atoms on the first line put either bohr or angstroem. boundary conditions, on second line, start with one of freebc, surface X lat 0 Z lat or periodic X lat Y lat Z lat. frozen positions, add a f, fy or fxz at the end of an atom line. input polarisation, add values at the end of an atom line. On output, forces are added for each element at the end of the XYZ file: 2 angstroem freebc Na Cl forces Na E Cl E-2 0 0

13 Performing a DFT calculation A self-consistent equation ρ(r) = Ψ i (r)ψ i (r), where ψ i satisfies i ( 12 ) 2 + V H [ρ] + V xc [ρ] + V ext + V pseudo ψ i = E i ψ i, (Kohn-Sham) DFT Ingredients A basis set for expressing the ψ i An potential, functional of the density several approximations exists (LDA,GGA,... ) A choice of the pseudopotential (if not all-electrons) (norm conserving, ultrasoft, PAW,... ) An (iterative) algorithm for finding the wavefunctions ψ i A (good) computer...

14 s in real space: Wavelet basis sets Properties of wavelet basis sets: localized both in real and in Fourier space allow for adaptivity (for core electrons) are a systematic basis set A basis set both adaptive and systematic, real space based x φ(x) ψ(x)

15 A brief description of wavelet theory Two kind of basis functions A Multi-Resolution real space basis The functions can be classified following the resolution level they span. Scaling Functions The functions of low resolution level are a linear combination of high-resolution functions = + m φ(x) = h j φ(2x j) j= m Centered on a resolution-dependent grid: φ j = φ 0 (x j).

16 A brief description of wavelet theory Wavelets They contain the DoF needed to complete the information which is lacking due to the coarseness of the resolution. φ(2x) = m j= m = h j φ(x j) + m j= m g j ψ(x j) Increase the resolution without modifying grid space SF + W = same DoF of SF of higher resolution ψ(x) = m j= m g j φ(2x j) All functions have compact support, centered on grid points.

17 Adaptivity of the mesh Adaptivity Resolution can be refined following the grid point. The grid is divided in: Data can thus be stored in compressed form. Localization property Low resolution pts (SF, 1 DoF) High resolution pts (SF + W, 8 DoF) Points of different resolution belong to the same grid. Empty regions must not be filled with basis functions. Optimal for big inhomogeneous systems, O(N) approach.

18 Adaptivity of the mesh Atomic positions (H 2 O), hgrid

19 Adaptivity of the mesh Fine grid (high resolution, frmult)

20 Adaptivity of the mesh Coarse grid (low resolution, crmult)

21 Defining the basis set in input.dft Main specific variables to control the basis set hgrid the grid size frmult the expansion around atoms for fine grid crmult the expansion around atoms for coarse grid Convergence properties Decreasing hgrid more degrees of freedom. Increasing crmult less box constraint. It is important to improve both parameters together to avoid error on one hiding improvement made on the other one.

22 input.dft: hgrid, crmult, frmult hgrid Step grid: twice as bigger than the lowest gaussian coefficient of pseudopotential Roughly 0.45 (only orthorhombic mesh) crmult Extension of the coarse resolution (factor 6.): depends on the extension of the HOMO for the atom frmult Extension of the fine resolution (factor 8.): depends on the pseudopotential coefficient In practice, we use only hgrid from 0.3 to 0.55.

23 Wavelets: No integration error Orthogonality, scaling relation dx φ k (x)φ j (x) = δ kj φ(x) = 1 2 m j= m h j φ(2x j) The hamiltonian-related quantities can be calculated up to machine precision in the given basis. The accuracy is only limited by the basis set (O ( h 14 grid) ) Exact evaluation of kinetic energy Obtained by convolution with filters: f(x) = c l φ l (x), 2 f(x) = c l φ l (x), l l c l = c j a l j, a l φ 0 (x) 2 xφ l (x), j

24 Wavelets: Separability in 3D The 3-dim scaling basis is a tensor product decomposition of 1-dim Scaling Functions/ Wavelets. φ e x,e y,e z j x,j y,j z (x,y,z) = φ e x j x (x)φ e y j y (y)φ e z j z (z) With (j x,j y,j z ) the node coordinates, φ (0) j and φ (1) j the SF and the W respectively. Gaussians and wavelets The separability of the basis allows us to save computational time when performing scalar products with separable functions (e.g. gaussians): Initial wavefunctions (input guess) Poisson solver Non-local pseudopotentials

25 Performing a DFT calculation A self-consistent equation ρ(r) = Ψ i (r)ψ i (r), where ψ i satisfies i ( 12 ) 2 + V H [ρ] + V xc [ρ] + V ext + V pseudo ψ i = E i ψ i, (Kohn-Sham) DFT Ingredients A basis set for expressing the ψ i An potential, functional of the density several approximations exists (LDA,GGA,... ) A choice of the pseudopotential (if not all-electrons) (norm conserving, ultrasoft, PAW,... ) An (iterative) algorithm for finding the wavefunctions ψ i A (good) computer...

26 Gaussian type separable s (HGH) Local part V loc (r) = Z [ ] [ ion r erf + exp 1 ( ) ] 2 r r 2rloc 2 r loc { ( ) 2 ( ) 4 ( r r r C 1 + C 2 + C 3 + C 4 r loc r loc Nonlocal (separable) part H( r, r ) H sep ( r, r ) = p l 1(r) = 2 r l+ 3 2 l 2 i=1 r loc Y s,m (ˆr) p s i (r) h s i p s i (r ) Ys,m(ˆr ) m + Y p,m (ˆr) p p 1 (r) hp 1 pp 1 (r ) Yp,m(ˆr ) m r l+ 7 2 l ) 6 } r l e 1 2 ( r ) 2 r l p Γ(l l r l+2 e 1 2 ( r ) 2 r l 2(r) = ) Γ(l + 7) 2

27 file (as psppar.n) No need if you use LDA or PBE functionals Default pseudopotentials inside! Put in the same directory as the calculation. Large library: Accurate and transferable

28 Performing a DFT calculation A self-consistent equation ρ(r) = Ψ i (r)ψ i (r), where ψ i satisfies i ( 12 ) 2 + V H [ρ] + V xc [ρ] + V ext + V pseudo ψ i = E i ψ i, (Kohn-Sham) DFT Ingredients A basis set for expressing the ψ i An potential, functional of the density several approximations exists (LDA,GGA,... ) A choice of the pseudopotential (if not all-electrons) (norm conserving, ultrasoft, PAW,... ) An (iterative) algorithm for finding the wavefunctions ψ i A (good) computer...

29 Exchange-Correlation Energy (input.dft) E xc [ρ] = K exact [ρ] K KS [ρ] + V e e exact [ρ] V Hartree [ρ] K KS [ρ]: Kinetic energy for a non-interaction electron gas Kohn-Sham formalism is exact (mapping of an interacting electron system into a non-interacting system) Many types from ABINIT: Use same codes (ixc: LDA=1, PBE=11, see manual) lib: code for exchange code for correlation ( see manual) Hybrid functionals from lib

30 Performing a DFT calculation A self-consistent equation ρ(r) = Ψ i (r)ψ i (r), where ψ i satisfies i ( 12 ) 2 + V H [ρ] + V xc [ρ] + V ext + V pseudo ψ i = E i ψ i, (Kohn-Sham) DFT Ingredients A basis set for expressing the ψ i An potential, functional of the density several approximations exists (LDA,GGA,... ) A choice of the pseudopotential (if not all-electrons) (norm conserving, ultrasoft, PAW,... ) An (iterative) algorithm for finding the wavefunctions ψ i A (good) computer...

31 Kohn-Sham Equations: Operators Apply different operators Having a one-electron hamiltonian in a mean field. [ 1 ] V eff (r)(r) ψ i = ε i ψ i V eff (r) = V ext (r) + V ρ(r ) r r dr + µ xc (r) E[ρ] can be expressed by the orthonormalized states of one particule: ψ i (r) with the fractional occupancy number f i (0 f i 1) : ρ(r) = f i ψ i (r) 2 i

32 Kohn-Sham Equations: Computing Energies Calculate different integrals E[ρ] = K [ρ] + U[ρ] U[ρ] = K [ρ] = m e i V dr f iψ i 2 ψ i dr V ext(r)ρ(r)+ 1 ρ(r)ρ(r ) dr dr V 2 V r r } {{ } Hartree We minimise with the variables ψ i (r) and f i with the constraint dr ρ(r) = N el. V + E xc [ρ] } {{ } exchange correlation

33 KS Equations: Self-Consistent Field Set of self-consistent equations: { 1 } V eff ψ i = ε i ψ i 2 m e with an effective potential: V eff (r) = V ext (r)+ ) dr ρ(r V r r } {{ } Hartree δe xc + δρ(r) } {{ } exchange correlation and: ρ(r) = 2 i f i ψ i (r) 2

34 KS Equations: Self-Consistent Field Set of self-consistent equations: { 1 } V eff ψ i = ε i ψ i 2 m e with an effective potential: V eff (r) = V ext (r)+ ) dr ρ(r V r r } {{ } Hartree δe xc + δρ(r) } {{ } exchange correlation and: ρ(r) = 2 i f i ψ i (r) 2 Poisson Equation: V Hartree = ρ (Laplacian: = 2 x y z 2 )

35 KS Equations: Self-Consistent Field Set of self-consistent equations: { 1 } V eff ψ i = ε i ψ i 2 m e with an effective potential: V eff (r) = V ext (r)+ ) dr ρ(r V r r } {{ } Hartree δe xc + δρ(r) } {{ } exchange correlation and: ρ(r) = 2 i f i ψ i (r) 2 Poisson Equation: V Hartree = ρ (Laplacian: = 2 x y z 2 )

36 KS Equations: Self-Consistent Field Set of self-consistent equations: { 1 } V eff ψ i = ε i ψ i 2 m e with an effective potential: V eff (r) = V ext (r)+ ) dr ρ(r V r r } {{ } Hartree δe xc + δρ(r) } {{ } exchange correlation and: ρ(r) = 2 i f i ψ i (r) 2 Poisson Equation: V Hartree = ρ (Laplacian: = 2 x y z 2 )

37 KS Equations: Self-Consistent Field Set of self-consistent equations: { 1 } V eff ψ i = ε i ψ i 2 m e with an effective potential: V eff (r) = V ext (r)+ ) dr ρ(r V r r } {{ } Hartree δe xc + δρ(r) } {{ } exchange correlation and: ρ(r) = 2 i f i ψ i (r) 2 Poisson Equation: V Hartree = ρ (Laplacian: = 2 x y z 2 )

38 KS Equations: Self-Consistent Field Set of self-consistent equations: { 1 } V eff ψ i = ε i ψ i 2 m e with an effective potential: V eff (r) = V ext (r)+ ) dr ρ(r V r r } {{ } Hartree δe xc + δρ(r) } {{ } exchange correlation and: ρ(r) = 2 i f i ψ i (r) 2 Poisson Equation: V Hartree = ρ (Laplacian: = 2 x y z 2 )

39 Direct Minimisation: Flowchart { } ψ i = caφ i a a Basis: adaptive mesh (0,...,N φ ) Orthonormalized

40 Direct Minimisation: Flowchart { } ψ i = caφ i a a inv FWT + Magic Filter Basis: adaptive mesh (0,...,N φ ) Orthonormalized occ ρ(r) = i ψ i (r) 2 Fine non-adaptive mesh: < 8N φ

41 Direct Minimisation: Flowchart { } ψ i = caφ i a a inv FWT + Magic Filter Basis: adaptive mesh (0,...,N φ ) Orthonormalized occ ρ(r) = i ψ i (r) 2 Fine non-adaptive mesh: < 8N φ Poisson solver V H (r) = V xc [ρ(r)] G(.)ρ V NL ({ψ i }) V effective

42 Direct Minimisation: Flowchart { } ψ i = caφ i a a inv FWT + Magic Filter Basis: adaptive mesh (0,...,N φ ) Orthonormalized occ ρ(r) = i ψ i (r) 2 Fine non-adaptive mesh: < 8N φ Poisson solver V H (r) = V xc [ρ(r)] G(.)ρ V NL ({ψ i }) V effective Kinetic Term

43 Direct Minimisation: Flowchart { } ψ i = caφ i a a inv FWT + Magic Filter Basis: adaptive mesh (0,...,N φ ) Orthonormalized occ ρ(r) = i ψ i (r) 2 Fine non-adaptive mesh: < 8N φ Poisson solver V H (r) = V xc [ρ(r)] G(.)ρ V NL ({ψ i }) FWT FWT δca i = E total c i (a) + Λ ij ca j j V effective Kinetic Term Λ ij =< ψ i H ψ j >

44 Direct Minimisation: Flowchart { } ψ i = caφ i a a inv FWT + Magic Filter Basis: adaptive mesh (0,...,N φ ) Orthonormalized occ ρ(r) = i ψ i (r) 2 Fine non-adaptive mesh: < 8N φ Poisson solver V H (r) = V xc [ρ(r)] G(.)ρ V NL ({ψ i }) FWT δca i = E total c i (a) + Λ ij ca j j c new,i a FWT preconditioning = c i a + h step δc i a V effective Kinetic Term Λ ij =< ψ i H ψ j > Steepest Descent, DIIS

45 Direct Minimisation: Flowchart { } ψ i = caφ i a a inv FWT + Magic Filter Basis: adaptive mesh (0,...,N φ ) Orthonormalized occ ρ(r) = i ψ i (r) 2 Fine non-adaptive mesh: < 8N φ Poisson solver V H (r) = V xc [ρ(r)] G(.)ρ V NL ({ψ i }) FWT Stop when δc i a small δca i = E total c i (a) + Λ ij ca j j c new,i a FWT preconditioning = c i a + h step δc i a V effective Kinetic Term Λ ij =< ψ i H ψ j > Steepest Descent, DIIS

46 input.dft: idsx DIIS history Use DIIS algorithm Direct Inversion in the Iterative Subspace Use idsx=6 previous iterations Memory-consuming (decrease if too big) Steepest descent if idsx=0 itermax=50 Specify maximum number of iterations Typically 15 iterations to converge gnrm_cv=1.e-4 convergence criterion Residue δψ 2 < gnrm_cv

47 Description of the file input.dft hx, hy, hz crmult, frmult, coarse and fine radius 1 ixc: parameter (LDA=1,PBE=11) ncharge: charge of the system, Electric field 1 0 nspin=1 non-spin polarization, total magnetic moment 1.E-04 gnrm_cv: convergence criterion gradient itermax, nrepmax 7 6 # CG (preconditioning), length of diis history 0 dispersion correction functional (0, 1, 2, 3) InputPsiId, output_wf, output_grid length of the tail, # tail CG iterations davidson treatment, # virtual orbitals, # plotted orbitals T disable the symmetry detection

48 Description of the file input.dft hx, hy, hz crmult, frmult, coarse and fine radius 1 ixc: parameter (LDA=1,PBE=11) ncharge: charge of the system, Electric field 1 0 nspin=1 non-spin polarization, total magnetic moment 1.E-04 gnrm_cv: convergence criterion gradient itermax, nrepmax 7 6 # CG (preconditioning), length of diis history 0 dispersion correction functional (0, 1, 2, 3) InputPsiId, output_wf, output_grid length of the tail, # tail CG iterations davidson treatment, # virtual orbitals, # plotted orbitals T disable the symmetry detection

49 Description of the file input.dft hx, hy, hz crmult, frmult, coarse and fine radius 1 ixc: parameter (LDA=1,PBE=11) ncharge: charge of the system, Electric field 1 0 nspin=1 non-spin polarization, total magnetic moment 1.E-04 gnrm_cv: convergence criterion gradient itermax, nrepmax 7 6 # CG (preconditioning), length of diis history 0 dispersion correction functional (0, 1, 2, 3) InputPsiId, output_wf, output_grid length of the tail, # tail CG iterations davidson treatment, # virtual orbitals, # plotted orbitals T disable the symmetry detection

50 Description of the file input.dft hx, hy, hz crmult, frmult, coarse and fine radius 1 ixc: parameter (LDA=1,PBE=11) ncharge: charge of the system, Electric field 1 0 nspin=1 non-spin polarization, total magnetic moment 1.E-04 gnrm_cv: convergence criterion gradient itermax, nrepmax 7 6 # CG (preconditioning), length of diis history 0 dispersion correction functional (0, 1, 2, 3) InputPsiId, output_wf, output_grid length of the tail, # tail CG iterations davidson treatment, # virtual orbitals, # plotted orbitals T disable the symmetry detection

51 Description of the file input.dft hx, hy, hz crmult, frmult, coarse and fine radius 1 ixc: parameter (LDA=1,PBE=11) ncharge: charge of the system, Electric field 1 0 nspin=1 non-spin polarization, total magnetic moment 1.E-04 gnrm_cv: convergence criterion gradient itermax, nrepmax 7 6 # CG (preconditioning), length of diis history 0 dispersion correction functional (0, 1, 2, 3) InputPsiId, output_wf, output_grid length of the tail, # tail CG iterations davidson treatment, # virtual orbitals, # plotted orbitals T disable the symmetry detection

52 Description of the file input.dft hx, hy, hz crmult, frmult, coarse and fine radius 1 ixc: parameter (LDA=1,PBE=11) ncharge: charge of the system, Electric field 1 0 nspin=1 non-spin polarization, total magnetic moment 1.E-04 gnrm_cv: convergence criterion gradient itermax, nrepmax 7 6 # CG (preconditioning), length of diis history 0 dispersion correction functional (0, 1, 2, 3) InputPsiId, output_wf, output_grid length of the tail, # tail CG iterations davidson treatment, # virtual orbitals, # plotted orbitals T disable the symmetry detection

53 Description of the file input.dft hx, hy, hz crmult, frmult, coarse and fine radius 1 ixc: parameter (LDA=1,PBE=11) ncharge: charge of the system, Electric field 1 0 nspin=1 non-spin polarization, total magnetic moment 1.E-04 gnrm_cv: convergence criterion gradient itermax, nrepmax 7 6 # CG (preconditioning), length of diis history 0 dispersion correction functional (0, 1, 2, 3) InputPsiId, output_wf, output_grid length of the tail, # tail CG iterations davidson treatment, # virtual orbitals, # plotted orbitals T disable the symmetry detection

54 Description of the file input.dft hx, hy, hz crmult, frmult, coarse and fine radius 1 ixc: parameter (LDA=1,PBE=11) ncharge: charge of the system, Electric field 1 0 nspin=1 non-spin polarization, total magnetic moment 1.E-04 gnrm_cv: convergence criterion gradient itermax, nrepmax 7 6 # CG (preconditioning), length of diis history 0 dispersion correction functional (0, 1, 2, 3) InputPsiId, output_wf, output_grid length of the tail, # tail CG iterations davidson treatment, # virtual orbitals, # plotted orbitals T disable the symmetry detection

55 Description of the file input.dft hx, hy, hz crmult, frmult, coarse and fine radius 1 ixc: parameter (LDA=1,PBE=11) ncharge: charge of the system, Electric field 1 0 nspin=1 non-spin polarization, total magnetic moment 1.E-04 gnrm_cv: convergence criterion gradient itermax, nrepmax 7 6 # CG (preconditioning), length of diis history 0 dispersion correction functional (0, 1, 2, 3) InputPsiId, output_wf, output_grid length of the tail, # tail CG iterations davidson treatment, # virtual orbitals, # plotted orbitals T disable the symmetry detection

56 Description of the file input.dft hx, hy, hz crmult, frmult, coarse and fine radius 1 ixc: parameter (LDA=1,PBE=11) ncharge: charge of the system, Electric field 1 0 nspin=1 non-spin polarization, total magnetic moment 1.E-04 gnrm_cv: convergence criterion gradient itermax, nrepmax 7 6 # CG (preconditioning), length of diis history 0 dispersion correction functional (0, 1, 2, 3) InputPsiId, output_wf, output_grid length of the tail, # tail CG iterations davidson treatment, # virtual orbitals, # plotted orbitals T disable the symmetry detection

57 Description of the file input.dft hx, hy, hz crmult, frmult, coarse and fine radius 1 ixc: parameter (LDA=1,PBE=11) ncharge: charge of the system, Electric field 1 0 nspin=1 non-spin polarization, total magnetic moment 1.E-04 gnrm_cv: convergence criterion gradient itermax, nrepmax 7 6 # CG (preconditioning), length of diis history 0 dispersion correction functional (0, 1, 2, 3) InputPsiId, output_wf, output_grid length of the tail, # tail CG iterations davidson treatment, # virtual orbitals, # plotted orbitals T disable the symmetry detection

58 Description of the file input.dft hx, hy, hz crmult, frmult, coarse and fine radius 1 ixc: parameter (LDA=1,PBE=11) ncharge: charge of the system, Electric field 1 0 nspin=1 non-spin polarization, total magnetic moment 1.E-04 gnrm_cv: convergence criterion gradient itermax, nrepmax 7 6 # CG (preconditioning), length of diis history 0 dispersion correction functional (0, 1, 2, 3) InputPsiId, output_wf, output_grid length of the tail, # tail CG iterations davidson treatment, # virtual orbitals, # plotted orbitals T disable the symmetry detection

59 s: wavelets self-consistent (SCF) Harris-Foulkes functional relativistic pseudopotential norm-conserving all electrons PAW Hybrid functionals beyond LDA GW non-relativistic [ N 3 scaling O(N) methods periodic non-periodic non-collinear collinear +V(r) non-spin polarized spin polarized LDA,GGA ] +µ xc (r) ψ k i = ε k i ψ k i atomic orbitals plane waves augmented real space LDA+U Gaussians Slater numerical finite difference Wavelet

60 Optimal for isolated systems 10 1 Test case: cinchonidine molecule (44 atoms) Plane waves Absolute energy precision (Ha) h = 0.4bohr E c = 40 Ha h = 0.3bohr Wavelets E c = 90 Ha Number of degrees of freedom E c = 125 Ha Allows a systematic approach for molecules Considerably faster than Plane Waves codes. 10 (5) times faster than ABINIT (CPMD) Charged systems can be treated explicitly with the same time

61 Systematic basis set Two parameters for tuning the basis The grid spacing hgrid The extension of the low resolution points crmult

62 Comparison with a code based on gaussian basis set The Daubechies expansion of a gaussian is known Results from gaussian codes can be imported in Gaussian code (CP2K) test, H 2 O and DZVP basis E [Ha] Input Guess Energy CP2K Imported energy converged energy 6-31G Conv energy Wavelet Conv energy with finite-size corrections crmult (basis extension)

63 using interpolating scaling function The Hartree potential is calculated in the interpolating scaling function basis. Poisson solver with interpolating scaling functions From the density ρ( j) on an uniform grid it calculates: ρ(r ) V H (r) = r r dr Very fast and accurate, optimal parallelization Can be used independently from the DFT code Integrated quantities (energies) are easy to extract Non-adaptive, needs data uncompression Explicitly free boundary conditions No need to subtract supercell interactions (charged systems).

64 for surface boundary conditions A Poisson solver for surface problems Based on a mixed reciprocal-direct space representation V px,p y (z) = 4π dz G( p ;z z )ρ px,p y (z), Can be applied both in real or reciprocal space codes No supercell or screening functions More precise than other existing approaches Example of the plane capacitor: Periodic Hockney Our approach V V V y y y x x x

65 input.geopt: or molecular dynamics DIIS 500 ncount_cluster: max steps during geometry relaxation 1.d0 1.d-3 frac_fluct: geometry optimization stops if force norm less than frac_fluct of noise maxval: maximum value of atomic forces 0.0d0 randdis: random amplitude for atoms 4.d0 5 betax: steepest descent step size, diis history Different algorithms: SDCG: Steepest Descent Conjugate Gradient DIIS: Direct Inversion in the iterative subspace VSSD: Variable Size Steepest Descent LBFGS: Limited BFGD AB6MD: Use molecular dynamics routines from ABINIT 6

66 input.geopt: or molecular dynamics DIIS 500 ncount_cluster: max steps during geometry relaxation 1.d0 1.d-3 frac_fluct: geometry optimization stops if force norm less than frac_fluct of noise maxval: maximum value of atomic forces 0.0d0 randdis: random amplitude for atoms 4.d0 5 betax: steepest descent step size, diis history Different algorithms: SDCG: Steepest Descent Conjugate Gradient DIIS: Direct Inversion in the iterative subspace VSSD: Variable Size Steepest Descent LBFGS: Limited BFGD AB6MD: Use molecular dynamics routines from ABINIT 6

67 input.geopt: or molecular dynamics DIIS 500 ncount_cluster: max steps during geometry relaxation 1.d0 1.d-3 frac_fluct: geometry optimization stops if force norm less than frac_fluct of noise maxval: maximum value of atomic forces 0.0d0 randdis: random amplitude for atoms 4.d0 5 betax: steepest descent step size, diis history Different algorithms: SDCG: Steepest Descent Conjugate Gradient DIIS: Direct Inversion in the iterative subspace VSSD: Variable Size Steepest Descent LBFGS: Limited BFGD AB6MD: Use molecular dynamics routines from ABINIT 6

68 input.geopt: or molecular dynamics DIIS 500 ncount_cluster: max steps during geometry relaxation 1.d0 1.d-3 frac_fluct: geometry optimization stops if force norm less than frac_fluct of noise maxval: maximum value of atomic forces 0.0d0 randdis: random amplitude for atoms 4.d0 5 betax: steepest descent step size, diis history Different algorithms: SDCG: Steepest Descent Conjugate Gradient DIIS: Direct Inversion in the iterative subspace VSSD: Variable Size Steepest Descent LBFGS: Limited BFGD AB6MD: Use molecular dynamics routines from ABINIT 6

69 input.geopt: or molecular dynamics DIIS 500 ncount_cluster: max steps during geometry relaxation 1.d0 1.d-3 frac_fluct: geometry optimization stops if force norm less than frac_fluct of noise maxval: maximum value of atomic forces 0.0d0 randdis: random amplitude for atoms 4.d0 5 betax: steepest descent step size, diis history Different algorithms: SDCG: Steepest Descent Conjugate Gradient DIIS: Direct Inversion in the iterative subspace VSSD: Variable Size Steepest Descent LBFGS: Limited BFGD AB6MD: Use molecular dynamics routines from ABINIT 6

70 input.geopt: or molecular dynamics DIIS 500 ncount_cluster: max steps during geometry relaxation 1.d0 1.d-3 frac_fluct: geometry optimization stops if force norm less than frac_fluct of noise maxval: maximum value of atomic forces 0.0d0 randdis: random amplitude for atoms 4.d0 5 betax: steepest descent step size, diis history Different algorithms: SDCG: Steepest Descent Conjugate Gradient DIIS: Direct Inversion in the iterative subspace VSSD: Variable Size Steepest Descent LBFGS: Limited BFGD AB6MD: Use molecular dynamics routines from ABINIT 6

71 High-Performance Computing: Massively parallel Two kinds of parallelisation By orbitals (Hamiltonian application, preconditioning) By components (overlap matrices, orthogonalisation) A few (but big) packets of data More demanding in bandwidth than in latency No need of fast network Optimal speedup (eff. 85%), also for big systems Cubic scaling code For systems bigger than 500 atomes (1500 orbitals) : orthonormalisation operation is predominant (N 3 )

72 Orbital distribution scheme Used for the application of the hamiltonian The hamiltonian (convolutions) is applied separately onto each wavefunction ψ 1 MPI 0 ψ 2 ψ 3 ψ 4 ψ 5 MPI 1 MPI 2

73 Coefficient distribution scheme Used for scalar product & orthonormalisation BLAS routines (level 3) are called, then result is reduced MPI 0 MPI 1 MPI 2 ψ 1 ψ 2 ψ 3 ψ 4 ψ 5 Communications are performed via MPI_ALLTOALLV

74 Performance in parallel () Distribution per orbitals (wavelets) Nothing to specify in input files

75 GPU-ported operations in (double precision) 20 Convolutions (OpenCL rewritten) GPU speedups between and 20 can be 6 obtained for different 4 sections 2 70 GPU speedup (Double prec.) locden locham precond Wavefunction size (MB) GPU speedup (Double prec.) DGEMM DSYRK Number of orbitals Linear algebra (CUBLAS library) The interfacing with CUBLAS is immediate, with considerable speedups

76 code on Hybrid architectures code can run on hybrid CPU/GPU supercomputers In multi-gpu environments, double precision calculations No Hot-spot operations Different code sections can be ported on GPU up to 20x speedup for some operations, 7x for the full parallel code Percent Seconds (log. scale) CPU code Hybrid code (rel.) Number of cores 1 Time (sec) Comm Other PureCPU precond locham locden LinAlg

77 input.perf file OpenCL acceleration: Add this line in in input.perf accel OCLGPU See the web page /Wiki/index.php?title=Input.perf for more information about all keywords (verbosity, input guess,...) associated to the performance of the code. The log file gives the list of possible keywords.

78 : Future developments self-consistent (SCF) Harris-Foulkes functional relativistic pseudopotential norm-conserving all electrons PAW Hybrid functionals beyond LDA GW non-relativistic [ N 3 scaling O(N) methods periodic non-periodic non-collinear collinear +V(r) non-spin polarized spin polarized LDA,GGA ] +µ xc (r) ψ k i = ε k i ψ k i atomic orbitals plane waves augmented real space LDA+U Gaussians Slater numerical finite difference Wavelet

79 Linear scaling version 16 min: 1000 atoms with the cubic scaling, 8000 atoms with the linear scaling one

80 Functionality: Mixed between physics and chemistry approach Order N (real space basis set as wavelets) Multi-scale approach (more than 1000 atoms feasible for a better parametrization) Numerical experience: One-day simulation Better exploration of atomic configurations Molecular Dynamics (4s per step for 32 water molecules)

81 Hands on See /Wiki/index.php?title=Category:Tutorials First runs with Basis-set convergence Acceleration example on different platforms: Kohn-Sham DFT Operation with GPU acceleration