High-performance computing of wind farms in the atmospheric boundary layer

Transcription

1 High-performance computing of wind farms in the atmospheric boundary layer VSC User Day Antwerp, 30 November 2015 Johan Meyers Mechanical Engineering, KU Leuven

2 Acknowledgements Research team Dries Allaerts, Pieter Bauweraerts, Vahid Bokharaie (postdoc), Juliaan Bossuyt, Ali Emre Yilmaz, Thomas Haas, Wim Munters, Cornelia Nita, Athanasios Vitsas, Liang Fang HPC support KU Leuven Martijn Oldenhof, Jan Ooghe, Leen Van Rentergem Funding ERC Active Wind Farms (Grant no ), FWO G , IDO/11/012 (BOF/KU Leuven) Computing: VSC TIER 1 and TIER 2 (KU Leuven)

3 London Array, UK 175 turbines 630 MW Source: London Array

4 Alta Wind Energy Center, USA 490 turbines 1,320 MW Source:

5 Gansu Wind Farm Project, China = cluster of wind farms Phase one (2010) More than 3500 turbines Installed capacity of 5,160 MW Total project (2020) Planned capacity of 20,000 MW Construction rate of 36 turbine/day One of six national megaprojects... Source:

6 Rotor Diameter and Rated Power IPCC (2011), Special report on renewable energy

7 Vestas V164 8MW (2014) Overall height of 220 m Rotor diameter of 164 m Source:

8 Horns Rev 1, DK 12 February 2008 Source:Vattenfall

9 Power deficit in wind farms Horns Rev Barthelmie et al., J. Phys.: Conference Series 75 (2007) Lillgrund Dahlberg JÅ, Vattenfal Tech Report, 2009

10 Outline 1. Single turbine: operation and Betz theory 2. Wind-farm simulations & the atmospheric boundary layer 3. SP-Wind: parallelization details 4. Perfomance issues and improving parallel communications 5. Summary and outlook

11 Single turbine: Betz theory Betz A. Das Maximum der theoretisch moglichen Ausnutzung des Windes durch Windmotoren. Zeitschrift für das gesamte Turbinenwesen 1920; 26: Joukowsky NE. Windmill of the NEJ type. In Transactions of the Central Institute for Aero-hydrodynamics of Moscow, 1920 (in Russian).

12 Single turbine: Betz theory /. % Wake mixing /. % Dissipation / % /. % /. %

13 Outline 1. Single turbine: operation and Betz theory 2. Wind-farm simulations & the atmospheric boundary layer 3. SP-Wind: parallelization details 4. Perfomance issues and improving parallel communications 5. Summary and outlook

14 Some physical aspects of ABLs Various ABL types Source: Stull, 1988, An introduction to Boundary Layer Meteorology Earth s rotation - Coriolis forces linear with velocity - Motions to pole are deflected to the right (northern hemisphere) Thermal stability - Stable stability surpresses vertical motions

15 Conventionally neutral ABL 3D view Free atmosphere Ground level

16 Wind-farm boundary layer modeling Classical approach Pressure-driven boundary layer Assumptions Turbines in inner layer Outer layer dynamics negligible Neglect Coriolis forces and thermal effects (for neutral ABL)

17 LES of wind-farm boundary layer Horns Rev Code: SP-Wind Mesh: 1536 x 768 x 192 Turbine model: ADM

18 Wind-farm boundary layer modeling Classical approach Pressure-driven boundary layer Assumptions Turbines in inner layer Outer layer dynamics negligible Neglect Coriolis forces and thermal effects (for neutral ABL) Does this approach still hold for Large turbines? Very large wind farms? What is the importance of outer layer effects?

19 LES of wind-farm boundary layer Outer layer effects z [km] z [km] y[km] x [km] z [km] y[km] x [km] Numerical details Wind farm: 20 x 9 turbines Resolution: 30 x 15 x 5 m Domain: 28.8 x 4.8 x 5 km Grid points: ca. 165M Simulated time: 4h y[km] x [km] High Performance Computing ca. 10 days on 512 cores

20 Wind-farm boundary layer: flow behaviour 1 km 500 m 200 m

21 Wind-farm optimal control Consider a wind-farm with N turbines AIM, min Subject to ;,, disk velocity F, 0 0 Degrees of freedom in control space, USE ADJOINT EQUATIOS TO CALCULATE GRADIENT (continuity) (momentum)

22 Forward and adjoint simulations Forward simulation Adjoint simulation

23 Optimal control of wind-farm boundary layer Power per Row Overall gain power output: 7% (preliminary further gains possible) Coordinated Optimal Control Hub height axial velocity Coordinated Optimal Control Greedy ( ) Greedy ( )

24 Outline 1. Single turbine: operation and Betz theory 2. Wind-farm simulations & the atmospheric boundary layer 3. SP-Wind: parallelization details 4. Perfomance issues and improving parallel communications 5. Summary and outlook

25 Numerical discretization Navier Stokes equations 0 Discretization in SP-Wind is using: o Fourier-spectral method in x, y o fourth-order energy-conserving finite differences in z

26 Numerical discretization In Fourier space (,, ) 0 Problem: convolution sum in Fourier space computational effort ~ Instead: Cost

27 Fourier transforms and parallelization Two-dimensional Fourier transforms (in x, y) Successive 1D (I)FFTs: first in x-direction, then in y-direction Use of FFTW Code parallelization (until end 2013): slab decomposition IFFT in y-dir MPI Communication IFFT in x-dir FFT in y-dir MPI Communication FFT in x-dir

28 New parallelization SP-Wind Pencil decomposition

29 New parallelization SP-Wind Fourier Space Real Space

30 Outline 1. Single turbine: operation and Betz theory 2. Wind-farm simulations & the atmospheric boundary layer 3. SP-Wind: parallelization details 4. Perfomance issues and improving parallel communications 5. Summary and outlook

31 Strong scaling initial tests Cores Challenging for parallelization

32 Strong scaling speed up Cores

33 Communication vs. no communication No communication Cores

34 Intra node versus internode communication First observations Important bottleneck: internode communication Internode communication << intranode communication

35 Intranode versus internode communicatoin Rearrange communication order Intranode Internode Internode Intranode Intranode

36 Improved communication arrangement Cores

37 Improve internode communication cores CASE: Probability density function MPI_ALLTOALL MPI_ALLTOALL_packed MPI_IALLTOALL FFTW_MPI samples for good convergence 0 0 0,1 0,2 0,3 0,4 0,5 0,6 Internode Communication time

38 Improve internode communication 1000 CASE: cores Probability density function MPI_ALLTOALL MPI_ALLTOALL_PACKED MPI_IALLTOALL FFTW_MPI 0 0,05 0,1 0,15 0,2 0,25 0,3 0,35 0,4 0,45 0,5 Internode Communication time

39 Improved performance SPEEDUP vs. 32 CORES CASE: Met intra node opt Beste versie Ideaal zonder intra node opt

40 HPC Case planning

41 Summary and outlook HPC simulations of wind-farm boundary layers o o Improve understanding of relevant physics (towards new generation of design models for industry) Change wind-farm control Supercomputing and parallelization o o Intranode communication much faster than internode Packing of communication can be advantageous (not always) Further improvements: towards cores per run

42 THANK YOU