Revision as of 09:53, 1 February 2022

Description

ECFD5 workshop logo.

• Event from 23th to 28th of January 2022
• Location: Centre Bonséjour, Merville-Franceville, near Caen (14)
• Two types of sessions:
  • common technical presentations: roadmaps, specific points.
  • mini-workshops. Potential workshops are listed below.
• Free of charge
• More than 50 participants from academics (CERFACS, CORIA, IMAG, LEGI, EM2C, UMONS, UVM, VUB, UCL, TUDelft), HPC center/experts (GENCI, AMD, CINES, CRIANN) and industry (Safran, Ariane Group, Siemens-Gamesa).

• Objectives
  • Bring together experts in high-performance computing, applied mathematics and multi-physics CFDs
  • Identify the technological barriers of exaflopic CFD via numerical experiments
  • Identify industrial needs and challenges in high-performance computing
  • Propose action plans to add to the development roadmaps of the CFD codes

News

• 03/11/2021: First announcement of the 5th Extreme CFD Workshop & Hackathon !

• 13/01/2022: After discussions with the participants, the 5th Extreme CFD Workshop & Hackathon is maintained as an in-person event! It will be also possible to attend to the plenary sessions and participate remotely to the workshop.

• 14/01/2022: The ECFD5 program is online! The plenary sessions will be announced soon!

• 20/01/2022: The plenary sessions are now defined:
  • P1 - 24/01/2022: GPU porting challenges and quantum computing, présentation machine Adastra by G. Staffelbach (CERFACS) + Presentation of the new cluster from CINES called Adastra by C. Andrieu (CINES)
  • P2 - 25/01/2022: News, perspectives and future of GPU computing applied to CFD by A. Toure (AMD)
  • P3 - 26/01/2022: Theory, applications and perspectives of the Lattice-Boltzmann Method by P. Boivin (M2P2)
  • P4 - 27/01/2022: Concepts and notions of mesh adaptation by C. Dapogny (LJK)

• 23/04/2022: The ECFD5 event has now started!! LinkedIn post

• 28/04/2022: The ECFD5 event is now finished! It was again a successful scientific week. See you next year for the ECFD6! LinkedIn post

Agenda

Thematics / Mini-workshops

These mini-workshops may change and cover more or less topics. This page will be adapted according to your feedback.

Combustion - K. Bioche, VUB

• Sub-project 1: H2/air jet-in-cross-flow numerical simulations (R. Le Dortz, E. Riber, Q. Douasbin)

The use of hydrogen as an aviation fuel requires new combustion chamber design. Among strategies to prevent flame flashback and low flame residence time, the micromix injection system is further studied by ENABLEH2. This systems corresponds to a multitude of H2/air jet-in-cross-flow configurations. A 3D numerical simulation with realistic thermodynamics and kinetics is now tractable thanks to massively parralel computing. This week saw the completion of the first steps towards the establishment of a complete simulation. (I) The non-reactive air injection in the combustion chamber. (II) The cross-injection of H2 without ignition. (III) The ignition of this mixture modeled with the skeletal kinetic mechanism of Boivin (H2, H, O2, OH, O, H2O, HO2, H2O2, N2). Further work will be realised concerning mesh refinement, modelling of NOx and porting of the computation on GPU.

• Sub-project 2: LES calculation of the MICADO test rig with multicomponent jetA1 (S. Puggelli, T. Lesaffre, E. Riber, B. Cuenot)

The EU-funded project ALTERNATE has the goal of exploring the possibility for a wider utilisation of aviation sustainable fuels. A part of the project deals with the assessment of the effect of SAFs on soot production: using the experimental information obtained at ONERA in high-pressure conditions on the MICADO test rig, the effect of Alcohol to Jet (ATJ-SPK) fuel on soot levels are assessed and compared with standard jet A1 emissions. During the project, STech and CERFACS are working jointly on the numerical modelling of soot emissions for jet-A1 and ATJ-SPK combustion in AVBP. Starting from the numerical setup under-development for jet-A1, the worshop permitted to: (I) Switch from a 2-step kinetic mechanism to a complex 29 species, 233 reacs and 15 QSS mechanism. This transition was efficiently conducted with the tool Multi Table Generator. (II) At this stage, an assessment of the effects of the flame sensor on the calculation results was carried out, indicating the consistent behaviour of a recently developped sensor w.r.t classical tools. (III) Switch towards a multicomponent formulation of jet-A1 and assessment of the effect of such advanced approach with respect to the single-component formulation previously employed. Further work will be realised to manage the stiffness of employed kinetics and to compare jet-A1 and ATJ-SPK fuels from a chemical point of view.

• Sub-project 3: Euler-Lagrange Multigrid Simulation (T. Lesaffre, O. Vermorel, E. Riber, B. Cuenot)

In Lagrange simulations, the point-source approach is based on a ponctual approximation of the particule and requires this last to be smaller than the mesh. The very fine meshes required to represent the Eulerian phase of Euler-Lagrange two-phase flow simulations can lead to a non-validity of the point-source hypothesis. This project aimed at implementing, in the AVBP solver, the simultaneous management and coupling of several simulations. During this week, the Eulerian and Lagrangian phase were successfuly computed on two different meshes and coupled via the CWIPI library. The good behaviour of this framework was assessed on a 1D Evaporation of kerosene droplets in an air stream test case. Encouraging preliminary performance results were obtained on a 3D injection case and require further work.

• Sub-project 4: Devolatilization modelling for biomass combustion (K. Bioche, L. Bricteux)

Biomass combustion simulations require the modelling of numerous physical phenomena: particle drying, devolatilization, gas-phase combustion, chars oxidation. Besides, the valorisation chains for biomass include fluidized bed reactors, fixed bed reactors and pulverized fuel burners. The Granular Flow Solver of YALES2 offers a good framework for the simulation of fluidized bed reactors and is functionnaly coupled with the reactive gas-phase solver of the same code. This week permitted to partically implement the modelling of devolatilization in this solver. A single-step kinetic scheme is considered for the particle mass evolution equation while the particle diameter evolves during the process. Further work is necessary to account for the thermal and mass couplings with the fluid phase.

• Sub-project 5: Thickened-Flame LES model in a Lattice-Boltzmann Method framework (P. Boivin, S. Zhao, M. Le Boursicaud)

The TFLES framework of the hybrid Lattice-Boltzmann sover ProLB was extended to account for recent sensor methods. During this week, a smooth flame sensor based on the curvature of the norm of the advancement variable gradient was developped. Also for filtering operations, the lattice requires to access data over three neighboring layers. A precise and continuous thickening factor was obtained with such method.

• Sub-project 6: NOx modeling applied to KIAI combustion chamber (J. Obando, P. Bénard, V. Moureau)

This project treated of the implementation of NOx modeling into simulations of the KIAI combustion chamber, experimentaly studied at CORIA lab. During this week, various NOx modeling strategies were listed. Associated kinetic mechanisms, among which analytical chemisty, were employed for 1D flame simulations in YALES2 solver. Further work include the use of such methods on the 3D computational case.

Static and dynamic mesh adaptation - G. Balarac, LEGI

Multi-phase flows - M. Cailler, SAFRAN TECH

• Sub-project 1: Hybrid E-E/E-L two-phase flow method (M. Cailler, F. Pecquery, I. El Yamani, V. Moureau)

The High-Fidelity approach based on ACLS & DMA allows a reliable description of interface dynamics. For design exploration, low-CPU methods with controlled level of fidelity are required. An interesting approach to reduce CPU cost relies on an hybrid approach based on an Eulerian representation of the gas and a Lagrangian description for the liquid phase. Objective of the ECFD5 was to explore the capability to reconstruct the interface normal of a liquid droplet made of particles on an Eulerian grid. First, a level-set based strategy relying on Geometric Multiple Markers Projection (Janodet et al., 2022) has been first tested showing good capabilities providing that the iso-surface distance equal 0 is well captured on the mesh. An alternative strategy based on the liquid volume fraction has been tested. This information was then used to build a velocity correction that is used to transport particles and ensures regularized particle positions. This preliminary benchmark work will be pursued on a liquid jet propagation problem.

• Sub-project 2: Jet atomization with a diffuse interface mathod (N. Odier, B. Péden, J. Carmona, P. Boivin)

A so-called diffuse interface method, using the multi-fluid formalism, coupled with a Riemann solver HLLC and a thermodynamic closure of type Noble-Abel Stiffened Gas (NASG), was implemented in AVBP during the thesis of Julien Carmona. During the ECFD5, the NASG thermodynamic was coupled with an improved HLLC solver implementation based on nodal information only, therefore leading to better performances in parallel. Moreover, the NSCBC terms expressed in the framework of the NASG thermodynamics were accurately derived. Future works include validation of the new solver capability on 1-D shock tube and isolated bubble test cases.

• Sub-project 3: Convergence of the interface curvature computation (G. Ghigliotti, J. Carmona, G. Balarac, G. Lartigue)

The computation of interface curvature in a level-set framework is based on the divergence of the gradient of the distance to the interface. This function being computed at 2nd order, one obtains a O(0) curvature, meaning that the error does not decrease with mesh refinement. We have implemented in YALES2 a strategy proposed by Emmanuel Maître and collaborators. This approach, implemented in a finite element code FreeFEM++, is based on the regularization (filtering) of the level-set gradient and curvature. This strategy has been tested for the simple test case of a static circular interface. We used two types of filters (simple gather-scatter or bilaplacian as developed by Lola Guedot (PhD thesis 2015)) on different mesh types (split quadrilaterals, isotropic triangular mesh, unstructured triangular mesh). The results are encouraging since a O(1) convergence is obtained in all cases. Further work is needed to tune the filter properties (amplitude and size) for different spatial resolutions and levelset "narrow band" width.

• Sub-project 4: Conservative two-fluid momentum transport (F. Pecquery, C. Merlin, M. Cailler, J. Carmona, V. Moureau)

The aim of this project was to investigate the applicability of the conservative two-fluid transport framework developed for passive scalars to the momentum conservation equation. First a complete two-fluid framework for the incompressible two-phase Navier-Stokes equations was proposed. Then the algorithms for discontinuous scalars (data extension in the other phase, reinitialization and transport) have been extended to discontinuous vectors. Moreover, some improvement of the data-structure were implemented to further generalize the framework and improve user-experience. To eventually conclude on the applicability of the two-fluid momentum approach, the proposed interfacial momentum flux model and correction step will be implemented in YALES2.

Numerics - G. Lartigue, CORIA

• Sub-project 1 : High order schemes for distorted meshes (M. Bernard, G. Balarac, G. Lartigue)

The high order framework (HOF) based on deconvolution recently developped and implemented in Yales2 permits to increase the accuracy of spatial numerical schemes on distorted meshes. Nevertheless, while dealing with highly anisotropic meshes (aspect-ratio above 10), the conditioning of the convolution matrix gets worse, complicating its inversion.

The aim of this project was too improve the robustness of the HO3 numerical scheme while dealing with highly anisotropic meshes. The strategy was to ensure third order accuracy as far as possible, then if conditioning gets too bad, to locally decrease the solution at 2nd order, using the HO2 convolution matrix which is unconditionally invertible. Then, solution must be reconstructed with care at control volume interface to account for this mixing between HO2 and HO3. Implementation has been successfully tested on transport of a scalar field in a non-uniform flow field with aspect-ratio up to 100.

• Sub-project 4: Avoiding the acoustic timestep restriction in two-phase flow simulations (V. Boniou, J. Paris, A. Vié, T. Schmitt, C. Tenaud, Y. Béchane)

In the context of simulating compressible two-phase flows with phase change, the CFL constraint related to the acoustic part of an explicit solver, especially in the liquid phase, can be prohibitive for evaporation studies implying large timescale separation between evaporation and acoustic wave propagation. The acoustic CFL constraint can be either relaxed by considering an implicit treatment of the acoustic part or avoided by deriving the low Mach limit of the multifluid model.

During this week, the first steps of this vast work have been initiated on single-phase flows by implementing the variable-density low Mach solver and the IMEX method applied on the Euler equations in TITAN.

The IMEX method was shown to provide degraded solutions in acoustic-driven cases, compared to conventional Riemann solutions. However, in transport-dominated cases, the increase of the transport CFL leads to a better numerical precision. A 2D cylindrical shock has been simulated with an acoustic CFL of 40 to assess the ability to run multidimensional compressible computations with large time step. The method was also extented to 4eq and 2nd order spatial accuracy.

Finally, the extension of the variable density solver to two-phase flows was performed using a VOF interface capturing method, considering a dilatable gas phase along with an incompressible liquid phase. Particular attention has been drawn to preserving liquid volume by treating the VOF transport accordingly with an adapted velocity field.

Turbulent flows - P. Bénard, CORIA

• Sub-project 1: Optimization of the actuator set for several wind turbines in YALES2 (F. Houtin Mongrolle, S. Gremmo, E. Muller, B. Duboc)

The implementation of the Actuator Line Method (ALM) into the YALES2 library leads to poor performances when many wind turbine rotors are set. Indeed, each rotor object is a derived type treated sequentially by all the processors participating to the computation. With 30 turbines in a computation, the return time is increased by 70% while the arithmetic intensity appears to be low. The objective of this sub-project is to improve the computation performances of the ALM already identified: (i) Assign one MPI communicator by rotor object gathering the processors close to the turbine and set-up a master/slave processus by communicator. This will allow the simultaneous rotors computation and reduce the number of MPI exchanges. (ii) Work on the domain decomposition to limit the number of processors attributed to each turbine. This would reduce or even eliminate MPI communications. (iii) Adapt the YALES2-BHawC aero-servo-elastic coupling to make it consistent with the new ALM implementation. During the workshop, the (i)-algorithm has been implemented with additional optimizations. Tests on a 30-turbine wind farm showed a 60% improvement compared to previous performances, which is very promising. The (ii)-feature still needs to be developed but should lead to the performances objective. The tests with the coupled code YALES2-BHawC were not conclusive yet and still require some work.

• Sub-project 2: Thermal effect in an atmospheric solver (U. Vigny, L. Voivenel, S. Zeoli, P. Benard)

Given the current environmental and energy challenges, maximising the wind farm electricity production is essential. Therefore, it becomes necessary to develop the most reliable and accurate prediction and simulation tools. Following this tenet, an atmospheric solver, which will take into account meteorological phenomena, should be developed. The preliminary work, going from bibliography study to road map was performed during the extreme cod workshop. Thus five parts have been identified: (I) The YALES2 Variable Density Solver (VDS) will be used because of the need to take into account buoyancy effect including for big density differences. (II) A wall law correction term, relative to atmospheric boundary layer will be added. (III) The actuator line method used to simulate wind turbine will be extended to VDS, modifying the velocity source term to a momentum source term. (IV) The Coriolis effects, depending on the latitude will be implemented. (V) The wall heat flux, allowing to simulate diurnal and nocturnal cycles on various terrains, is more realistic than a target wall temperature. From this work, future development are now clear and just waiting to be developed.

• Sub-project 3: Dynamic stall correction models for Horizontal Axis Wind Turbine (S. Gremmo, E. Muller, B. Duboc, F. Houtin-Mongrolle)

The Actuator Line Method implementation into YALES2 library suffers from the lack of correction models for some 3D unsteady effects. An important one, the dynamic stall, implies an unsteady modification of the aerodynamic loads, i.e. the polars, with the angle of attack. With the standard ALM model, static 2D polars are used. Adding a dynamic stall correction model allows to have an hysteresis loop on the evolution of the forces coefficients with the angle of attack. Several models exist in the literature: Oye, BeddoesLeishman, Sheng, Risoe, ONERA... During the workshop, the Oye model was selected, because of its simplicity, and implemented. First, new ingredients necessary to the model were added in the polars look-up table generator used to describe the blades. Then, the Oye correction was integrated into the YALES2 library. Finally, the model was tested on simple configurations. It is now essential to further validate the implementation.

• Sub-project4: Immersed Boundary solvers uniformisation into YALES2 (I. Tsetoglou, M. Cailler, P. Benez, S. Mendez, P. Benard)

A novel Conservative Lagrangian Immersed Boundary (CLIB) method [Tsetoglou et al., 2021] has been developed by CORIA & Safran Tech for wind turbines & gearbox lubrication simulations. Nevertheless, two different implementations of the same method were created that tends to diverge. This sub-project aims to develop a shared numerical kernel (CLIB) on which 2 solvers are based: CIB for aerodynamic applications (ICS + CLIB) and TPS for two-phase flows (SPS+CLIB). The newly developed kernel and solvers were tested and validated on test cases: 2D flows around static/rotating/oscillating disk, 3D flow around a rotating cylinder and 2D flows around counter-rotating ellipses. The newt steps involve the continuation of the validation of both solvers and its documentation, as well as the development of wall model for immersed bodies.

• Sub-project 5: TBLE wall model for LES with pressure gradient on a simple turbomachinery geometry (M. Cizeron, N. Odier, R. Vicquelin)

Wall modeling is often used in LES to alleviate the computational cost that would be required to resolve all the length scales up to the solid boundaries of the domain. The classical way of doing it is by using an algebraic model to provide the wall friction and heat flux, with a coupling to the LES solver at the first off-wall nodes. The wall model was designed from analyzing RANS equation with strong assumptions such as planar flow, equilibrium and no pressure gradient. These assumptions are often far from true in real applications, such as turbomachinery applications, where the use of a wall model is mandatory due to the size of the calculation. During this workshop, a wall model relying on the resolution of the Thin Boundary Layer Equations (TBLE) was studied, which had been implemented by EM2C. The addition of a pressure gradient to these equations has been conducted and tested, at first only for the 1D wall model solver, then on a 3D turbulent channel. It remains to be tested on a diffuser configuration with a real pressure gradient to quantify the effect of the new wall model. The influence of the point considered to do the coupling between the LES and the wall model (ie. its distance to the wall) has also been tested both for the TBLE and the original algebraic model, showing that coupling farther from the wall yields better results and reduces the so-called log-layer mismatch.

• Sub-project 6: Tools for rough wall modelling (A. Barge, S. Meynet)

Within the STREAM project framework, a roughness-resolved Large-Eddy Simulation (RRLES) database is being built. The aim of this latter is to be representative of rough channel flows, especially for additive-manufacturing heat exchangers. First RRLES have already been performed. From turbulence and rough wall stress statistics analysis of the results, a first stochastic model, which reproduces the statistical behavior of the wall stress vector, have been proposed. The modeled wall stress allows a better prediction of the pressure drop in a flat wall channel compared to the use of the mean value of the wall stress measured in RRLES alone. However, the near wall region is still mispredicted and the model is correlated in time but not in space. The aim of this ECFD5 was to develop tools to improve modelling and explore new ways. A roughness mapping tool for smooth surfaces have been implemented into YALES2 to get local surface height. This tool is based on an existing in-house surface roughness generator developed for the STREAM project. The idea is to use the map to generated space correlated fluctuations for the wall shear stress. Some bugs still remain to fully use this tool. In parallel, the modelling approach was extended to passive scalar, especially for temperature. To this end, new random tools as white noise, unit sphere random walk and Gaussian / Log-normal stochastic processes have been coded. Finally, the idea of using walls as velocity source terms emerged during this ECFD5. The principle is to mask a grid layer above the wall and to transport the rough map on this grid to estimate the roughness effects above the wall. Parametrizing and testing these tools remained to be done at the end of ECFD5.

User experience - J. Leparoux, SAFRAN TECH

• Sub-project 1: Multi combustion model chemtable generator (S. Dillon, R. Mercier)

Due to the high computational cost of LES of turbulent combustion with detailed chemistry, tabulated chemistry approaches remain a popular choice within the combustion community. This popularity of tabulated chemistry leads to research into the development of novel combustion models for which a platform to test these models is required. The objective of the workshop was to therefore begin developing an easy-to-use chemistry table generator within Python. Given that there exists numerous turbulent combustion models (F-TACLES Diffusion, F-TACLES Premixed, 2PFT, etc.), a generalised multi-model code was necessary. Along with multi-model capabilities, the code was required to function with multiple solvers for the creation of the flame database (REGATH and CANTERA). With these objectives in mind, the code “TabulatEd Chemistry GenERator for Aeronautical CombusTion” (TECERACT) was created. Advancements were made in all key areas using code already developed by Safran Tech. TECERACT includes a parallel flame database generator and current developments are focused on multi-model functionality and simplified mesh management within Python. Once finished, the TECERACT code will provide a user-friendly chemistry table generator capable of producing chemistry tables for LES simulations and its multi-model structure will serve as a platform for future turbulent combustion model development/testing.

• Sub-project 2: Task-driven automatic run sequence (R. Mercier, J. Leparoux, M. Cailler, R. Letournel)

The principal objective of this project was to develop a new simulation workflow based on Task-driven approach. This approach could offer a higher flexibility to manage CFD simulations by decomposing the whole simulation on small sequences (run initialization, fuel injection, etc.)

This week showed the achievement of the first steps towards a complete task-driven simulation. (i) The whole simulation workflow was set-up (ii) Missing actions were implemented allowing to update simulation status on-the-fly (from non reactive to reactive simulation) (iii) State are now embedded from the restart solution allowing to restart from the last known state. Further work is still needed to obtain the full implementation of the AMC framework based on task-driven approach. Especially to automate the adaptation loop

• Sub-project 3: YALES2 Tools & Gitlab CI (J. Leparoux, A. Tstetoglou)

• Sub-project 4: Wind energy tools (E. Muller, S. Gremmo, F. Houtin-Mongrolle, B. Duboc)

Set-up phase of simulations involving several wind turbines is very demanding and error-prone, especially for simulations carried out with the coupled solver YALES2-BHawC*. This type of simulation can involve several dozen input files and then, an assistant tool is required.

The objectives of this projet were (i) develop a python tools to set all necessary ACTUATOR_SET objects (ii) provide basic visualization plots allowing to monitor simulations on-the-fly (by post-processing YALES2 temporals and BHawC results files).

*BHawC : aero-servo-elastic solver used and developped at Siemens Gamesa Renewable Energy for wind turbine design and certification

Hackathon - G. Staffelbach, CERFACS

• Participants: I. d'Ast, J. Legaux, G. Staffelbach, P. Begou, G. Lartigue, V. Moureau, A. Toure, C. Laurie, S. Delamare, C. Andrieu, C. Jourdain

AMD GPU hardware is still relatively unknown in our CFD community. This hackathon was the opportunity to deep dive into the AMD dev environment to prepare the arrival of AdAstra at CINES. Both YALES and AVBP have been ported to the AOMP framework using ROCm 4.5 on the GRID5000 Neowise system. CPU execution posed no issues and we were able to focus on GPU Offloading using OpenMP. On the YALES2 side, a mini-app encompassing the typical YALES2 structure hierarchy and loop execution was extracted from the code to evaluate different porting strategies and on the AVBP side the current OpenACC GPU offloading was translated to OpenMP focusing on the viscosity computation kernel. We learnt that the current supported standard of OpenMP in ROCm 4.5 does not allow for direct offloading of reference values inside an derived type structure but is was possible to use aliases such as pointers or flat array copies to do the job. This should be solved with the support of OpenMP 5.0 Another troublesome issues, was the lack of support for offloading of array vector operations (ex : array(:) = 1.0 ) rendering the explicitation of the loops for these manadatory.

Some bugs remain and it is encouraged to use the latest compiler version when working on the porting ( the release of flang 14.0.1 saved us a lot of time as we had started with 14.0.0 ). Offloading of the miniapp of YALES2 yielded a times 60 acceleration of the kernel whereas the offloading of the viscosity model in a full avbp simulation yielded an 7 times factor in performance when comparing on core to one GPU. These results are to be taken with a grain of salt but are really encouraging.

For the next steps, a porting strategy for both codes will be implemented (depending on the OpenMP 5 support ) and discussions are underway with CINES and other partners so as to offer the best experience to both code's communities on AdAstra at its release.