OpenFOAM open source CFD on ANSELM

Transcription

1 OpenFOAM Tomáš Brzobohatý Supercomputing for industry SC4Industry Research Programme 3 - Libraries for Parallel Computing, VŠB-TU Ostrava

2 OpenFOAM - Introduction - Features - OpenFOAM Case Outline OpenFOAM on ANSELM - Running applications - Running applications in parallel - Compiling applications and libraries Benchmarks

3 What OpenFOAM is? OpenFOAM - Introduction The OpenFOAM (Open Field Operation and Manipulation) CFD Toolbox based on C++. OpenFOAM is free and open source software, released under the GNU General Public License. OpenFOAM includes over 80 solver applications that simulate specific problems in engineering mechanics and over 170 utility applications that perform pre- and post-processing tasks, e.g. meshing, data visualization, etc. Friendly syntax for partial differential equations Unstructured polyhedral grid capabilities Automatic parallelization of applications written using OpenFOAM high-level syntax Commercial support and training provided by the developers

4 OpenFOAM - Introduction OpenFOAM syntax for tensor operations and partial differential equations ρu t For example the equation is represented by the code: + φu- μ U = p Solve ( ); fvm::ddt(rho,u) + fvm::div(phi,u) - fvm::laplacian(mu,u) == - fvc::grad(p)

5 History OpenFOAM - Introduction The original development of OpenFOAM started in the late 1980s at Imperial College, London. The FOAM, was sold by UK company Nabla Ltd. before being released as open source in On 15 August 2011, OpenCFD announced its acquisition by Silicon Graphics International (SGI). On September 12, 2012, the ESI Group announced the acquisition of OpenFOAM Ltd from SGI. Where we can find info? Homepage - Wiki - CFD Forum -

6 OpenFOAM Features

7 Incompressible flows OpenFOAM - Features adjointshapeoptimizationfoam icofoam MRFSimpleFoam nonnewtonianicofoam pimpledymfoam pimplefoam pisofoam poroussimplefoam simplefoam SRFSimpleFoam

8 Incompressible flows OpenFOAM - Features adjointshapeoptimizationfoam icofoam MRFSimpleFoam nonnewtonianicofoam pimpledymfoam pimplefoam pisofoam poroussimplefoam simplefoam SRFSimpleFoam Multiphase flows bubblefoam cavitatingfoam compressibleinterfoam interfoam interdymfoam intermixingfoam interphasechangefoam LTSInterFoam MRFInterFoam MRFMultiphaseInterFoam multiphaseinterfoam porousinterfoam settlingfoam twoliquidmixingfoam twophaseeulerfoam simplefoam SRFSimpleFoam

9 Compressible flows OpenFOAM - Features rhocentralfoam rhocentraldymfoam rhopimplefoam rhoporousmrfltspimplefoam rhoporoussimplefoam rhoporousmrfpimplefoam rhosimplecfoam rhosimplefoam sonicdymfoam sonicliquidfoam

10 Compressible flows OpenFOAM - Features rhocentralfoam rhocentraldymfoam rhopimplefoam rhoporousmrfltspimplefoam rhoporousmrfsimplefoam rhoporousmrfpimplefoam rhosimplecfoam rhosimplefoam sonicdymfoam sonicliquidfoam chemfoam coldenginefoam dieselenginefoam dieselfoam enginefoam firefoam PDRFoam reactingfoam Combustion rhoreactingfoam XiFoam

11 Compressible flows OpenFOAM - Features rhocentralfoam rhocentraldymfoam rhopimplefoam rhoporousmrfltspimplefoam rhoporousmrfsimplefoam rhoporousmrfpimplefoam rhosimplecfoam rhosimplefoam sonicdymfoam sonicliquidfoam chemfoam coldenginefoam dieselenginefoam dieselfoam enginefoam firefoam PDRFoam reactingfoam Combustion rhoreactingfoam XiFoam Conjugate Heat Transfer buoyantbafflesimplefoam buoyantboussinesqpimplefoam buoyantboussinesqsimplefoam buoyantpimplefoam buoyantsimplefoam chtmultiregionfoam

12 RAS Turbulence Models OpenFOAM - Features laminar kepsilon komega komegasst RNGkEpsilon NonlinearKEShih LienCubicKE qzeta LaunderSharmaKE LamBremhorstKE LienCubicKELowRe LienLeschzinerLowRe LaunderGibsonRSTM realizableke SpalartAllmaras

13 RAS Turbulence Models OpenFOAM - Features laminar kepsilon komega komegasst RNGkEpsilon NonlinearKEShih LienCubicKE qzeta LaunderSharmaKE LamBremhorstKE LienCubicKELowRe LienLeschzinerLowRe LaunderGibsonRSTM realizableke SpalartAllmaras LES Turbulence Models laminar Smagorinsky Smagorinsky2 dynsmagorinsky dynlagrangian scalesimilarity mixedsmagorinsky dynmixedsmagorinsky komegasstsas oneeqeddy dynoneeqeddy locdynoneeqeddy specteddyvisc LRDDiffStress DeardorffDiffStress SpalartAllmaras SpalartAllmarasDDES SpalartAllmarasIDDES

14 Transport models OpenFOAM - Features Newtonian CrossPowerLaw BirdCarreau HerschelBulkley powerlaw interfaceproperties

15 Transport models OpenFOAM - Features Newtonian CrossPowerLaw BirdCarreau HerschelBulkley powerlaw interfaceproperties ThermoProperties icopolynomial perfectgas hconstthermo janafthermo speciethermo consttransport polynomialtransport sutherlandtransport hpolynomialthermo

16 Transport models OpenFOAM - Features Newtonian CrossPowerLaw BirdCarreau HerschelBulkley powerlaw interfaceproperties ThermoProperties icopolynomial perfectgas hconstthermo janafthermo speciethermo consttransport polynomialtransport sutherlandtransport hpolynomialthermo Radiation Models P1 fvdom viewfactor

17 Transport models OpenFOAM - Features Newtonian CrossPowerLaw BirdCarreau HerschelBulkley powerlaw interfaceproperties ThermoProperties icopolynomial perfectgas hconstthermo janafthermo speciethermo consttransport polynomialtransport sutherlandtransport hpolynomialthermo Radiation Models Reaction Models P1 fvdom viewfactor hpsimixturethermo hspsimixturethermo hrhomixturethermo hsrhomixturethermo homogeneousmixture inhomogeneousmixture veryinhomogeneousmixture dieselmixture basicmulticomponentmixture multicomponentmixture reactingmixture egrmixture

18 Mesh Generation OpenFOAM - Features blockmesh extrudemesh extrude2dmesh extrudetoregionmesh snappyhexmesh

19 Mesh Generation OpenFOAM - Features blockmesh extrudemesh extrude2dmesh extrudetoregionmesh snappyhexmesh Mesh Manipulation checkmesh mergemeshes movemesh polydualmesh refinemesh renumbermesh splitmesh transformpoints autorefinemesh modifymesh refinehexmesh refinementlevel refinewalllayer

20 Mesh Generation OpenFOAM - Features blockmesh extrudemesh extrude2dmesh extrudetoregionmesh snappyhexmesh Mesh Conversion Mesh Manipulation checkmesh mergemeshes movemesh polydualmesh refinemesh renumbermesh splitmesh transformpoints autorefinemesh modifymesh refinehexmesh refinementlevel refinewalllayer ansystofoam cfx4tofoam fluentmeshtofoam foammeshtofluent foamtostarmesh gambittofoam gmshtofoam ideasunvtofoam mshtofoam netgenneutraltofoam plot3dtofoam star4tofoam tetgentofoam writemeshobj

21 Pre-Processing OpenFOAM - Features applyboundarylayer applywallboundaryconditions changedictionary mapfields setfields viewfactorsgen

22 Pre-Processing OpenFOAM - Features applyboundarylayer applywallboundaryconditions changedictionary mapfields setfields viewfactorsgen Post-Processing foamdatatofluent foamtoensight foamtoensightparts foamtogmv foamtotecplot360 foamtovtk Co Lambda2 Mach

23 OpenFOAM case OpenFOAM case Constant - contains a full description of the case mesh in a subdirectory polymesh and files specifying physical properties for the application concerned, e.g.transportproperties. System - for setting parameters associated with the solution procedure itself. Time directories - containing individual files of data for particular fields Processors - directories are named processorn where N=0,1 represents a processor number and contains a time directory, containing the decomposed field descriptions, and a constant/polymesh directory containing the decomposed mesh description. properties polymesh constant Points Faces Cells boundary processors* case system controldict fvschemes fvsolution decomposepardict 0 time directories > 0

24 OpenFOAM on ANSELM

25 Available engineering modules ~]$ module avail /opt/modules/modulefiles/engineering ansys/14.5.x matlab/r2013a-edu comsol/43b-com openfoam/2.2.1-gcc481-openmpi1.6.5-dp comsol/43b-edu openfoam/2.2.1-gcc481-openmpi1.6.5-sp lsdyna/7.x.x openfoam/2.2.1-icc-openmpi1.6.5-dp matlab/r2013a-com To create OpenFOAM environment on ANSELM give the commands: ~]$ module load openfoam/2.2.1-icc-openmpi1.6.5-dp ~]$ source $FOAM_BASHRC

26 Getting Started Create a project directory within the $HOME/OpenFOAM directory named <USER> and create a directory named run within it, e.g. by typing: [brz021@login2.anselm ~]$ mkdir -p $FOAM_RUN Project directory is now available by typing [brz021@login2.anselm ~]$ cd /home/brz021/openfoam/brz /run or [brz021@login2.anselm ~]$ cd $FOAM_RUN Copy the tutorial examples directory in the OpenFOAM distribution to the run directory [brz021@login2.anselm ~]$ cp -r $FOAM_TUTORIALS $FOAM_RUN Run the first example case of incompressible laminar flow in a cavity: [brz021@login2.anselm ~]$ cd $FOAM_RUN/tutorials/incompressible/icoFoam/cavity [brz021@login2.anselm ~]$ blockmesh [brz021@login2.anselm ~]$ icofoam

27 Running Serial Applications Create a Bash script: pokus.sh #!/bin/bash module load openfoam/2.2.1-icc-openmpi1.6.5-dp source $FOAM_BASHRC # source to run functions. $WM_PROJECT_DIR/bin/tools/RunFunctions cd $FOAM_RUN/tutorials/incompressible/icoFoam/cavity runapplication blockmesh runapplication icofoam Job Submission $ qsub -A OPEN-0-0 -q qprod -l select=1:ncpus=1,walltime=03:00:00 pokus.sh

28 OpenFOAM on ANSELM Running applications in parallel

29 Running applications in parallel #!/bin/bash #PBS N motorbike #PBS l select=2:ncpus=16 #PBS l walltime=01:00:00 #PBS q qprod #PBS A IT4I-1-4 pokusparallel.pbs module load openfoam/2.2.1-icc-openmpi1.6.5-dp source $FOAM_BASHRC cd $FOAM_RUN/tutorials/incompressible/simpleFoam/motorBike nproc = 32 mpirun hostfile ${PBS_NODEFILE} np $nproc snappyhexmesh overwrite parallel tee snappyhexmesh.log mpirun hostfile ${PBS_NODEFILE} np $nproc potentialfoam -nofunctionobject-writep parallel tee potentialfoam.log mpirun hostfile ${PBS_NODEFILE} np $nproc simplefoam parallel tee simplefoam.log

30 Running applications in parallel decomposepardict The mesh and fields are decomposed using the decomposepar utility. constant case system properties polymesh Points Faces Cells boundary 0 controldict fvschemes fvsolution decomposepardict time directories > 0 processors*

31 Running applications in parallel numberofsubdomains 32; method simple; decomposepardict simplecoeffs { n ( ); delta 0.001; } hierarchicalcoeffs { n ( ); delta 0.001; order xyz; } manualcoeffs { datafile ""; }

32 Running applications in parallel simple - simple geometric decomposition in which the domain is split into pieces by direction hierarchical - hierarchical geometric decomposition which is the same as simple except the user specifies the order in which the directional split is done scotch - scotch decomposition which requires no geometric input from the user and attempts to minimise the number of processor boundaries. manual - manual decomposition, where the user directly specifies the allocation of each cell to a particular processor.

33 Running applications in parallel Create a Bash script: pokus.sh #!/bin/bash module load openfoam/2.2.1-icc-openmpi1.6.5-dp source $FOAM_BASHRC # source to run functions. $WM_PROJECT_DIR/bin/tools/RunFunctions cd $FOAM_RUN/tutorials/incompressible/simpleFoam/motorBike runapplication blockmesh runapplication decomposepar Job Submission $ qsub -A OPEN-0-0 -q qprod -l select=1:ncpus=1,walltime=03:00:00 pokus.sh $ qsub pokusparallel.pbs

34 OpenFOAM on ANSELM Compile an own solvers

35 Compile an own solver Initialize OpenFOAM environment before compiling your solver ~]$ module load openfoam/2.2.1-icc-openmpi1.6.5-dp ~]$ source $FOAM_BASHRC $ cd $FOAM_RUN/ Create directory applications/solvers in user directory $ mkdir p applications/solvers $ cd applications/solvers Copy icofoam solver s source files $ cp r $FOAM_SOLVERS/incompressible/icoFoam/ My_icoFoam $ cd My_icoFoam Rename icofoam.c to My_icoFOAM.C $ mv icofoam.c My_icoFoam.C

36 Compile an own solver Edit files file in Make directory: icofoam.c EXE = $(FOAM_APPBIN)/icoFoam My_icoFoam.C EXE = $(FOAM_USER_APPBIN)/My_icoFoam In directory My_icoFoam give the compilation command: $ wmake

37 BENCHMARKS

38 Boeing 787 Dreamliner Benchmarks

39 Boeing 787 Dreamliner Benchmarks M Cells M Unknowns Mesh generated by snappyhexmesh simplefoam k-omegasst 2000 time step iter. solver parameters p U k omega - GAMG - smoothsolver - smoothsolver - smoothsolver Tolerance 1e-7 relativetolerance 0.1

40 Boeing 787 Dreamliner Benchmarks

41 Lamborghini Aventador Benchmarks

42 Lamborghini Aventador M Cells M Unknowns Mesh generated by snappyhexmesh simplefoam k-omegasst 2000 time step iter. Benchmarks solver parameters p U k omega - GAMG - smoothsolver - smoothsolver - smoothsolver Tolerance 1e-8 relativetolerance 0.001

43 Lamborghini Aventador Benchmarks

44 Hydraulic Valve Benchmarks 52.3 M Cells Mesh generated by ANSYS Workbench simplefoam k-omegasst 1000 time step iter.

45 Hydraulic Valve Benchmarks

46 Time [s] Hydraulic Valve Benchmarks N Cells [M]

47 Thermal Sensor Benchmarks 5.3 M Cells Mesh generated by ANSYS Workbench chtmultiregionsimplefoam k-epsilon 1000 time step iter. 11 regions

48 Thermal Sensor Benchmarks

49 Lamp Benchmarks 17.3 M Cells Mesh generated by ANSYS Workbench chtmultiregionsimplefoam fvdom Radiation Model laminar time step iter. 6 Regions

50 Lamp Benchmarks

51 Lamborghini Aventador Benchmarks Transient solver - pisofoam 23 M Cells ncorrectors 2 k-omegasst solver parameters p - PCG U - PBiCG k - PBiCG omega - PBiCG

52 Lamborghini Aventador Benchmarks time steps solution time - 40 hr 256 cores

53 VSB Campus Benchmarks

54 VSB Campus Benchmarks 18 M Cells Mesh generated by snappyhexmesh simplefoam k-epsilon 2000 time step iter.

55 VSB Campus Benchmarks

56 snappyhexmesh

57 Stonavka - Olše Benchmarks

58 Stonavka - Olše Benchmarks

59 snappyhexmesh Benchmarks VSB Campus 1,51 1,01 0,51 0,

60 Boeing 787 Dreamliner Benchmarks snappyhexmesh scalability N cores Time [h]

61 Lamborghini Aventador Benchmarks snappyhexmesh scalability N cores Time [h]

62 Supermarine Spitfire Benchmarks

63 VW Beetle Benchmarks

64 Other Open Source Benchmarks Elmer Code Aster Code Saturne Palabos SU 2

65 benchmarket.it4i.cz Benchmarks