Difference between revisions of "Ecfd:ecfd 5th edition"

From Extreme CFD workshop
Jump to: navigation, search
(User experience - J. Leparoux, SAFRAN TECH)
 
(68 intermediate revisions by 19 users not shown)
Line 12: Line 12:
 
* Free of charge
 
* Free of charge
 
* More than 50 participants from academics (CERFACS, CORIA, IMAG, LEGI, EM2C, UMONS, UVM, VUB, UCL, TUDelft), HPC center/experts (GENCI, AMD, CINES, CRIANN) and industry (Safran, Ariane Group, Siemens-Gamesa).
 
* More than 50 participants from academics (CERFACS, CORIA, IMAG, LEGI, EM2C, UMONS, UVM, VUB, UCL, TUDelft), HPC center/experts (GENCI, AMD, CINES, CRIANN) and industry (Safran, Ariane Group, Siemens-Gamesa).
 +
 +
* Objectives
 +
** Bring together experts in high-performance computing, applied mathematics and multi-physics CFDs
 +
** Identify the technological barriers of exaflopic CFD via numerical experiments
 +
** Identify industrial needs and challenges in high-performance computing
 +
** Propose action plans to add to the development roadmaps of the CFD codes
  
 
== News ==
 
== News ==
Line 21: Line 27:
 
* 13/01/2022: After discussions with the participants, the '''5th Extreme CFD Workshop & Hackathon''' is maintained as an in-person event!  It will be also possible to attend to the plenary sessions and participate remotely to the workshop.  
 
* 13/01/2022: After discussions with the participants, the '''5th Extreme CFD Workshop & Hackathon''' is maintained as an in-person event!  It will be also possible to attend to the plenary sessions and participate remotely to the workshop.  
  
* 14/01/2022: The ECFD5 program is online ! The plenary sessions will be announced soon!
+
* 14/01/2022: The [[#Agenda|ECFD5 program]] is online! The plenary sessions will be announced soon!
  
[[File:ECFD5_program.png|text-bottom|600px]]
+
* 20/01/2022: The plenary sessions are now defined:
 +
** P1 - 24/01/2022: GPU porting challenges and quantum computing, présentation machine Adastra by G. Staffelbach (CERFACS) + Presentation of the new cluster from CINES called Adastra by C. Andrieu (CINES)
 +
** P2 - 25/01/2022: News, perspectives and future of GPU computing applied to CFD by A. Toure (AMD)
 +
** P3 - 26/01/2022: Theory, applications and perspectives of the Lattice-Boltzmann Method by P. Boivin (M2P2)
 +
** P4 - 27/01/2022: Concepts and notions of mesh adaptation by C. Dapogny (LJK)
  
== Objectives ==
+
* 23/04/2022: '''The ECFD5 event has now started!!''' [https://www.linkedin.com/feed/update/urn:li:activity:6891053385072594944| LinkedIn post]
  
* Bring together experts in high-performance computing, applied mathematics and multi-physics CFDs
+
* 28/04/2022: '''The ECFD5 event is now finished!''' It was again a successful scientific week. See you next year for the '''ECFD6'''! [https://www.linkedin.com/posts/l%C3%A9a-voivenel-642ab7186_avbp-yales2-yales2-activity-6892778892801716224-3zxn| LinkedIn post]
* Identify the technological barriers of exaflopic CFD via numerical experiments
+
 
* Identify industrial needs and challenges in high-performance computing
+
[[File:ECFD5_PhotoGroupe.jpeg|text-bottom|600px]]
* Propose action plans to add to the development roadmaps of the CFD codes
+
  
 
== Agenda ==
 
== Agenda ==
  
 
+
[[File:ECFD5_program.png|text-bottom|600px]]
  
 
== Thematics / Mini-workshops ==
 
== Thematics / Mini-workshops ==
  
 
These mini-workshops may change and cover more or less topics. This page will be adapted according to your feedback.
 
These mini-workshops may change and cover more or less topics. This page will be adapted according to your feedback.
 +
 +
== Projects ==
 +
 +
=== Hackathon - G. Staffelbach, CERFACS ===
 +
* '''Participants: I. d'Ast, J. Legaux, G. Staffelbach, P. Begou, G. Lartigue, V. Moureau, A. Toure, C. Laurie, S. Delamare, C. Andrieu, C. Jourdain'''
 +
While hybrid CPU/GPU computing has seen growing interest in the CFD community, AMD GPU hardware is a newcomer for this application. This hackathon was the opportunity to deep dive into the AMD development environment and to prepare the arrival of AdAstra at CINES, a new super-computer which relies on AMD CPU and GPU hardware. Two flagship codes of the applied CFD community, namely YALES2 and AVBP, have been ported to the AOMP framework using ROCm 4.5 on the GRID5000 Neowise system. This machine [https://www.grid5000.fr/w/GENCI-Neowise_Usage] was donated by AMD to GENCI and INRIA to help in fighting the COVID pandemic and it is operated by the Grid'5000 infrastructure.
 +
 +
Part of the project was spent in porting the codes to the new development environment. This work posed no real issues and CPU execution was achieved fast so that we were able to focus on GPU Offloading using OpenMP.
 +
On the YALES2 side, a mini-app encompassing the typical YALES2 structure hierarchy and loop execution was extracted from the code to evaluate different porting strategies. On the AVBP side, the current OpenACC GPU offloading was translated to OpenMP focusing on the viscosity computation kernel. We learnt that the current supported standard of OpenMP in ROCm 4.5 does not allow for direct offloading of reference values inside a derived type structure but is was possible to use aliases such as pointers or flat array copies to offload. This should be solved with the support of OpenMP 5.0. Another troublesome issues, was the lack of support for offloading of array vector operations  (ex : array(:) = 1.0 ) rendering the explicitation of the loops for these manadatory.
 +
 +
Some bugs remain and it is encouraged to use the latest compiler version when working on the porting (the release of flang 14.0.1 saved us a lot of time as we had started with 14.0.0). Offloading of the miniapp of YALES2 yielded a times 60 acceleration of the kernel whereas the offloading of the viscosity model in a full avbp simulation yielded an 7 times factor in performance when comparing on core to one GPU. These results are to be taken with a grain of salt but are really encouraging. For the next steps, a porting strategy for both codes will be implemented (depending on the OpenMP 5 support) and discussions are underway with CINES and other partners so as to offer the best experience to both code's communities on AdAstra at its release.
  
 
=== Combustion - K. Bioche, VUB  ===
 
=== Combustion - K. Bioche, VUB  ===
  
 +
* '''Sub-project 1: H2/air jet-in-cross-flow numerical simulations (R. Le Dortz, E. Riber, Q. Douasbin)'''
 +
 +
The use of hydrogen as an aviation fuel requires new combustion chamber design. Among strategies to prevent flame flashback and low flame residence time, the micromix injection system is further studied by ENABLEH2. This systems corresponds to a multitude of H2/air jet-in-cross-flow configurations. A 3D numerical simulation with realistic thermodynamics and kinetics is now tractable thanks to massively parralel computing. This week saw the completion of the first steps towards the establishment of a complete simulation. (I) The non-reactive air injection in the combustion chamber. (II) The cross-injection of H2 without ignition. (III) The ignition of this mixture modeled with the skeletal kinetic mechanism of Boivin (H2, H, O2, OH, O, H2O, HO2, H2O2, N2). Further work will be realised concerning mesh refinement, modelling of NOx and porting of the computation on GPU.
 +
 +
* '''Sub-project 2: LES calculation of the MICADO test rig with multicomponent jetA1 (S. Puggelli, T. Lesaffre, E. Riber, B. Cuenot)'''
 +
 +
The EU-funded project ALTERNATE has the goal of exploring the possibility for a wider utilisation of aviation sustainable fuels. A part of the project deals with the assessment of the effect of SAFs on soot production: using the experimental information obtained at ONERA in high-pressure conditions on the MICADO test rig, the effect of Alcohol to Jet (ATJ-SPK) fuel on soot levels are assessed and compared with standard jet A1 emissions. During the project, STech and CERFACS are working jointly on the numerical modelling of soot emissions for jet-A1 and ATJ-SPK combustion in AVBP. Starting from the numerical setup under-development for jet-A1, the worshop permitted to: (I) Switch from a 2-step kinetic mechanism to a complex 29 species, 233 reacs and 15 QSS mechanism. This transition was efficiently conducted with the tool Multi Table Generator. (II) At this stage, an assessment of the effects of the flame sensor on the calculation results was carried out, indicating the consistent behaviour of a recently developped sensor w.r.t classical tools. (III) Switch towards a multicomponent formulation of jet-A1 and assessment of the effect of such advanced approach with respect to the single-component formulation previously employed. Further work will be realised to manage the stiffness of employed kinetics and to compare jet-A1 and ATJ-SPK fuels from a chemical point of view.
 +
 +
* '''Sub-project 3: Euler-Lagrange Multigrid Simulation (T. Lesaffre, O. Vermorel, E. Riber, B. Cuenot)'''
 +
 +
In Lagrange simulations, the point-source approach is based on a ponctual approximation of the particule and requires this last to be smaller than the mesh. The very fine meshes required to represent the Eulerian phase of Euler-Lagrange two-phase flow simulations can lead to a non-validity of the point-source hypothesis. This project aimed at implementing, in the AVBP solver, the simultaneous management and coupling of several simulations. During this week, the Eulerian and Lagrangian phase were successfuly computed on two different meshes and coupled via the CWIPI library. The good behaviour of this framework was assessed on a 1D Evaporation of kerosene droplets in an air stream test case. Encouraging preliminary performance results were obtained on a 3D injection case and require further work.
 +
 +
* '''Sub-project 4: Devolatilization modelling for biomass combustion (K. Bioche, L. Bricteux)'''
 +
 +
Biomass combustion simulations require the modelling of numerous physical phenomena: particle drying, devolatilization, gas-phase combustion, chars oxidation. Besides, the valorisation chains for biomass include fluidized bed reactors, fixed bed reactors and pulverized fuel burners. The Granular Flow Solver of YALES2 offers a good framework for the simulation of fluidized bed reactors and is functionnaly coupled with the reactive gas-phase solver of the same code. This week permitted to partically implement the modelling of devolatilization in this solver. A single-step kinetic scheme is considered for the particle mass evolution equation while the particle diameter evolves during the process. Further work is necessary to account for the thermal and mass couplings with the fluid phase.
 +
 +
* '''Sub-project 5: Thickened-Flame LES model in a Lattice-Boltzmann Method framework (P. Boivin, S. Zhao, M. Le Boursicaud)'''
 +
 +
The TFLES framework of the hybrid Lattice-Boltzmann sover ProLB was extended to account for recent sensor methods. During this week, a smooth flame sensor based on the curvature of the norm of the advancement variable gradient was developped. Also for filtering operations, the lattice requires to access data over three neighboring layers. A precise and continuous thickening factor was obtained with such method.
 +
 +
* '''Sub-project 6: NOx modeling applied to KIAI combustion chamber (J. Obando, P. Bénard, V. Moureau)'''
 +
 +
This project treated of the implementation of NOx modeling into simulations of the KIAI combustion chamber, experimentaly studied at CORIA lab. During this week, various NOx modeling strategies were listed. Associated kinetic mechanisms, among which analytical chemisty, were employed for 1D flame simulations in YALES2 solver. Further work include the use of such methods on the 3D computational case.
  
 
=== Static and dynamic mesh adaptation - G. Balarac, LEGI ===
 
=== Static and dynamic mesh adaptation - G. Balarac, LEGI ===
  
 +
* '''Sub-project 1: Anisotropic mesh adaptation in YALES2 (A. Grenouilloux, A. Stock, G. Lartigue, G. Balarac, A. Froehly, C. Dapogny, J. Leparoux)'''
  
=== Multi-phase flows - M. Cailler, SAFRAN TECH ===
+
Mesh adaptation based on isotropic meshes is now a mature tool in YALES2 based on the Mmg library. This parallel mesh adaptation method is very effective for increasing the resolution in sensitive regions of the flow even when using billion-cell meshes. In many configurations, a significant gain in terms of computational cost can be expected through the use of anisotropic mesh. However, this requires to address several aspects: (i) the ability to generate anisotropic meshes via the coupling between YALES2 and Mmg, (ii) to define an anisotropic metric adapted to the simulated flow, and (iii) to assess the properties of the numerical methods (prediction/stability) of YALES2 on such meshes. This project has addressed the two first points (see sub-project 5, in Numerics for the last one). During the workshop, a full anisotropic mesh adaptation strategy has been implemented in YALES2. Anisotropic mesh adaptation needs to prescribe a metric tensor to define the ellipsoid characterizing the anisotropic tetrahedral element. First developments were then devoted to the manipulation and the control of this metric tensor (metric intersection, graduation control, aspect ratio control, etc...). Second part of the project was devoted to the determination of an anisotropic mesh based on computed fields. The Hessian-based metric definition has been implemented for this purpose. Successfull demonstrations have been conducted for 2D and 3D meshes during the workshop paving the way to upcoming applications.
  
 +
=== Multi-phase flows - M. Cailler, SAFRAN TECH  ===
 +
 +
* '''Sub-project 1: Hybrid E-E/E-L two-phase flow method (M. Cailler, F. Pecquery, I. El Yamani, V. Moureau)'''
 +
 +
The High-Fidelity approach based on ACLS & DMA allows a reliable description of interface dynamics. For design exploration, low-CPU methods with controlled level of fidelity are required. An interesting approach to reduce CPU cost relies on an hybrid approach based on an Eulerian representation of the gas and a Lagrangian description for the liquid phase. Objective of the ECFD5 was to explore the capability to reconstruct the interface normal of a liquid droplet made of particles on an Eulerian grid. First, a level-set based strategy relying on Geometric Multiple Markers Projection (Janodet et al., 2022) has been first tested showing good capabilities providing that the iso-surface distance equal 0 is well captured on the mesh. An alternative strategy based on the liquid volume fraction has been tested. This information was then used to build a velocity correction that is used to transport particles and ensures regularized particle positions. This preliminary benchmark work will be pursued on a liquid jet propagation problem.
 +
 +
* '''Sub-project 2: Jet atomization with a diffuse interface mathod (N. Odier, B. Péden, J. Carmona, P. Boivin)'''
 +
 +
A so-called diffuse interface method, using the multi-fluid formalism, coupled with a Riemann solver HLLC and a thermodynamic closure of type Noble-Abel Stiffened Gas (NASG), was implemented in AVBP during the thesis of Julien Carmona. During the ECFD5, the NASG thermodynamic was coupled with an improved HLLC solver implementation based on nodal information only, therefore leading to better performances in parallel. Moreover, the NSCBC terms expressed in the framework of the NASG thermodynamics were accurately derived. Future works include validation of the new solver capability on 1-D shock tube and isolated bubble test cases.
 +
 +
* '''Sub-project 3: Convergence of the interface curvature computation (G. Ghigliotti, J. Carmona, G. Balarac, G. Lartigue)'''
 +
 +
The computation of interface curvature in a level-set framework is based on the divergence of the gradient of the distance to the interface. This function being computed at 2nd order, one obtains a O(0) curvature, meaning that the error does not decrease with mesh refinement.
 +
We have implemented in YALES2 a strategy proposed by Emmanuel Maître and collaborators. This approach, implemented in a finite element code FreeFEM++, is based on the regularization (filtering) of the level-set gradient and curvature.
 +
This strategy has been tested for the simple test case of a static circular interface.
 +
We used two types of filters (simple gather-scatter or bilaplacian as developed by Lola Guedot (PhD thesis 2015)) on different mesh types (split quadrilaterals, isotropic triangular mesh, unstructured triangular mesh).
 +
The results are encouraging since a O(1) convergence is obtained in all cases.
 +
Further work is needed to tune the filter properties (amplitude and size) for different spatial resolutions and levelset "narrow band" width.
 +
 +
* '''Sub-project 4: Conservative two-fluid momentum transport (F. Pecquery, C. Merlin, M. Cailler, J. Carmona, V. Moureau)'''
 +
 +
The aim of this project was to investigate the applicability of the conservative two-fluid transport framework developed for passive scalars to the momentum conservation equation. First a complete two-fluid framework for the incompressible two-phase Navier-Stokes equations was proposed. Then the algorithms for discontinuous scalars (data extension in the other phase, reinitialization and transport) have been extended to discontinuous vectors. Moreover, some improvement of the data-structure were implemented to further generalize the framework and improve user-experience. To eventually conclude on the applicability of the two-fluid momentum approach, the proposed interfacial momentum flux model and correction step will be implemented in YALES2.
  
 
=== Numerics - G. Lartigue, CORIA ===
 
=== Numerics - G. Lartigue, CORIA ===
  
 +
* '''Sub-project 1 : High order schemes for distorted meshes (M. Bernard, G. Balarac, G. Lartigue)'''
 +
The high order framework (HOF) based on deconvolution recently developed and implemented in Yales2 permits to increase the accuracy of spatial numerical schemes on distorted meshes.
 +
Nevertheless, while dealing with highly anisotropic meshes (aspect-ratio above 10), the conditioning of the convolution matrix gets worse, complicating its inversion.
 +
 +
The aim of this project was too improve the robustness of the HO3 numerical scheme while dealing with highly anisotropic meshes.
 +
The strategy was to ensure third order accuracy as far as possible, then if conditioning gets too bad, to locally decrease the solution at 2nd order, using the HO2 convolution matrix which is unconditionally invertible.
 +
Then, solution must be reconstructed with care at control volume interface to account for this mixing between HO2 and HO3.
 +
Implementation has been successfully tested on transport of a scalar field in a non-uniform flow field with aspect-ratio up to 100.
 +
 +
* '''Sub-project 2: A coupling between YALES2 solver and a VPM method for the simulation of external flows (P. Billuart, G. Lartigue, P. Bénard)'''
 +
Numerical simulations of flows past immersed objects are ubiquitous in both research and industry, spanning e.g. vehicle aero- or hydrodynamics, biological propulsion, wind energy, civil engineering, etc. Such problems are typically solved using either a (1) body-fitted grid solver, like the YALES2 finite volumes solver, or via (2) a vortex method, like the VPM (Vortex Particle Mesh) solver. Vortex methods are indeed quite appealing in that problem because they yield very low dispersion and diffusion errors which puts them among the most efficient methods to accurately advect vortical structures over long distances. In addition, their vorticity-based formulation accommodates quite naturally the enforcement of the unbounded conditions of external flows. However, vortex methods are much less suited to capture boundary layers than classical body-fitted grid solvers, e.g. finite elements, finite volumes, etc, because of their isotropic computational elements. This leads to the conclusion that one could exploit the advantages of vortex methods and body fitted-grid solvers in a coupled approach; using YALES2 for solving the near-wall part of the flow, while solving the wake using the VPM method.
 +
This project aimed to implement such a coupling using the CWIPI library. It requires two information exchanges between both solvers; (a) an interpolation of vorticity and velocity VPM fields onto the outer YALES2 boundary and, (b) the interpolation of the YALES2 velocity field onto the VPM grid. While tests regarding the MPI communicator splitting, CWIPI initializations and CWIPI interpolation initializations demonstrate a correct implementation of those, they also point out that a bug still remains in the interpolation process from VPM to YALES2. 
 +
 +
The next steps will be: (1) keeping on identifying this bug, (2) implementing the information exchange from YALES2 to VPM and (3) synchronizing both solvers on the global communicator via MPI calls.
 +
 +
* '''Sub-Project 3: Assessment of the YALES2 compressible flow solvers (CPS,ECS,SIC) for turbomachinery applications (P. Tene, L. Bricteux, S. Lavagnoli, Y. Béchane, P. Bénez)'''
 +
High propulsive efficiency calls for Ultra-High By-Pass ratio geared turbofans. A huge fan ensuring the high dilution is rotating slowly. This fan is driven by a low-pressure turbine (LPT) rotating fast (diminishing stage count and weight). Both components are linked using a gearbox. As the fan produces 80% of the thrust, the LPT that drives it is a key component to achieve efficiency. LPT operates at transonic exit Mach numbers and low-Reynolds numbers (60-80k). Within this range of operating conditions, there is a critical shortage of aerodynamic and performance measurements.
 +
 +
This week we assessed the ability of the YALES2 Explicit compressible flow solver (ECS) to ensure consistent results for such applications. The first results were very promising, and proposals were made (e.g imposing total pressure as a boundary condition) to improve the development of this solver. In addition, an assessment of the different numerical schemes of this solver in terms of diffusion has also been carried out. Advances have also been made in the development of the YALES2 semi-implicit compressible solver (SIC). This solver will eventually (in the context of this application), improve the computational cost because unlike ECS, the time step here is not limited by the acoustic CFL.
 +
 +
* '''Sub-project 4: Avoiding the acoustic timestep restriction in two-phase flow simulations (V. Boniou, J. Paris, A. Vié, T. Schmitt, C. Tenaud, Y. Béchane)'''
 +
In the context of simulating compressible two-phase flows with phase change, the CFL constraint related to the acoustic part of an explicit solver, especially in the liquid phase, can be prohibitive for evaporation studies implying large timescale separation between evaporation and acoustic wave propagation.
 +
The acoustic CFL constraint can be either relaxed by considering an implicit treatment of the acoustic part or avoided by deriving the low Mach limit of the multifluid model.
 +
 +
During this week, the first steps of this vast work have been initiated on single-phase flows by implementing the variable-density low Mach solver and the IMEX method applied on the Euler equations in TITAN.
 +
 +
The IMEX method was shown to provide degraded solutions in acoustic-driven cases, compared to conventional Riemann solutions. However, in transport-dominated cases, the increase of the transport CFL leads to a better numerical precision. A 2D cylindrical shock has been simulated with an acoustic CFL of 40 to assess the ability to run multidimensional compressible computations with large time step. The method was also extented to 4eq and 2nd order spatial accuracy.
 +
 +
Finally, the extension of the variable density solver to two-phase flows was performed using a VOF interface capturing method, considering a dilatable gas phase along with an incompressible liquid phase. Particular attention has been drawn to preserving liquid volume by treating the VOF transport accordingly with an adapted velocity field.
 +
 +
* '''Sub-project 5 : Numerical schemes for anisotropic meshes (R. Letournel, G. Lartigue, G. Balarac, M. Bernard)'''
 +
Many efforts are devoted to the development of mesh adaptation based on anisotropic meshes in Yales2. For many configurations (e.g. shocks, mixing layers, near wall flows), a significant gain in terms of computational cost can be expected through the use of such meshes. The aim of this project was to characterize Yales2 numerical schemes on anisotropic meshes in comparison with isotropic meshes.
 +
 +
The strategy was at first to test different Laplacian operators on analytic functions (linear, quatratic, bump in one or two directions) in direct sense (apply the Laplacian operator) and indirect (solve the Poisson equation), to verify the order of convergence of the operators. Different meshes were considered: Isomesh, Quad2tri, MMG meshes, with different stretching (corresponding to anisotropic ratios). This exhaustive study showed that only the SIMPLEX operator remains of order 2 for the Poisson solver, while the IGNORE_SKEWNESS and PAIR_BASED operators do not converge.
 +
 +
In a second step, the performance of the incompressible solver on a 2D Poiseuille flow have been evaluated for increasingly stretched meshes. The coarsening in the preferential direction of the flow allows to decrease the convective time step and thus to reduce the RTTS (Return Time To Solution), while keeping the same error. These conclusions are valid for anisotropic ratios of the order of 10, beyond which the time step no longer decreases and further studies are needed to understand the limitations in these cases.
 +
 +
* '''Sub-project 6 : Poisson solver comparison for a two-phase flow CFD code (J. C. Brändle de Motta, A. Poux, T. Ménard)'''
 +
Poisson solvers known to be highly sensitive to discontinuities and represent the main computational costs of incompressible codes.
 +
Archer is a mature two-phase flow CFD code. Its historical Poisson solver is currently based on a Conjugate gradient with multigrid preconditioner (MGCG).
 +
The aim of this project was to couple Archer and the PETSc library, assess its performances and challenge the historical solver of Archer.
 +
 +
The coupling with PETSc has been implemented during the workshop and the comparison with MGCG has been done up to 2 thousands cores in single and two phase turbulent flow.
 +
Multiple PETSc configuration has been tested, and show that MGCG is very competitive (one order of magnitude faster than PETSc)
  
 
=== Turbulent flows - P. Bénard, CORIA ===
 
=== Turbulent flows - P. Bénard, CORIA ===
 +
* '''Sub-project 1: Optimization of the actuator set for several wind turbines in YALES2 (F. Houtin Mongrolle, S. Gremmo, E. Muller, B. Duboc)'''
 +
The implementation of the Actuator Line Method (ALM) into the YALES2 library leads to poor performances when many wind turbine rotors are set. Indeed, each rotor object is a derived type treated sequentially by all the processors participating to the computation. With 30 turbines in a computation, the return time is increased by 70% while the arithmetic intensity appears to be low. The objective of this sub-project is to improve the computation performances of the ALM already identified:
 +
(i) Assign one MPI communicator by rotor object gathering the processors close to the turbine and set-up a master/slave processus by communicator. This will allow the simultaneous rotors computation and reduce the number of MPI exchanges.
 +
(ii) Work on the domain decomposition to limit the number of processors attributed to each turbine. This would reduce or even eliminate MPI communications.
 +
(iii) Adapt the YALES2-BHawC aero-servo-elastic coupling to make it consistent with the new ALM implementation.
 +
During the workshop, the (i)-algorithm has been implemented with additional optimizations. Tests on a 30-turbine wind farm showed a 60% improvement compared to previous performances, which is very promising. The (ii)-feature still needs to be developed but should lead to the performances objective. The tests with the coupled code YALES2-BHawC were not conclusive yet and still require some work. 
  
 +
* '''Sub-project 2: Thermal effect in an atmospheric solver (U. Vigny, L. Voivenel, S. Zeoli, P. Benard)'''
 +
Given the current environmental and energy challenges, maximising the wind farm electricity production is essential. Therefore, it becomes necessary to develop the most reliable and accurate prediction and simulation tools. Following this tenet, an atmospheric solver, which will take into account meteorological phenomena, should be developed. The preliminary work, going from bibliography study to road map was performed during the extreme cod workshop. Thus five parts have been identified:
 +
(I) The YALES2 Variable Density Solver (VDS) will be used because of the need to take into account buoyancy effect including for big density differences.
 +
(II) A wall law correction term, relative to atmospheric boundary layer will be added.
 +
(III) The actuator line method used to simulate wind turbine will be extended to VDS, modifying the velocity source term to a momentum source term.
 +
(IV) The Coriolis effects, depending on the latitude will be implemented.
 +
(V) The wall heat flux, allowing to simulate diurnal and nocturnal cycles on various terrains, is more realistic than a target wall temperature.
 +
From this work, future development are now clear and just waiting to be developed.
  
=== Compressible - L. Bricteux, UMONS ===
+
* '''Sub-project 3: Dynamic stall correction models for Horizontal Axis Wind Turbine (S. Gremmo, E. Muller, B. Duboc, F. Houtin-Mongrolle)'''
 +
The Actuator Line Method implementation into YALES2 library suffers from the lack of correction models for some 3D unsteady effects. An important one, the dynamic stall, implies an unsteady modification of the aerodynamic loads, i.e. the polars, with the angle of attack. With the standard ALM model, static 2D polars are used. Adding a dynamic stall correction model allows to have an hysteresis loop on the evolution of the forces coefficients with the angle of attack. Several models exist in the literature: Oye, BeddoesLeishman, Sheng, Risoe, ONERA... During the workshop, the Oye model was selected, because of its simplicity, and implemented. First, new ingredients necessary to the model were added in the polars look-up table generator used to describe the blades. Then, the Oye correction was integrated into the YALES2 library. Finally, the model was tested on simple configurations. It is now essential to further validate the implementation. 
  
 +
* '''Sub-project4: Immersed Boundary solvers uniformisation into YALES2 (I. Tsetoglou, M. Cailler, P. Benez, S. Mendez, P. Benard)'''
 +
A novel Conservative Lagrangian Immersed Boundary (CLIB) method [Tsetoglou et al., 2021] has been developed by CORIA & Safran Tech for wind turbines & gearbox lubrication simulations. Nevertheless, two different implementations of the same method were created that tends to diverge. This sub-project aims to develop a shared numerical kernel (CLIB) on which 2 solvers are based: CIB for aerodynamic applications (ICS + CLIB) and TPS for two-phase flows (SPS+CLIB). The newly developed kernel and solvers were tested and validated on test cases: 2D flows around static/rotating/oscillating disk, 3D flow around a rotating cylinder and 2D flows around counter-rotating ellipses. The newt steps involve the continuation of the validation of both solvers and its documentation, as well as the development of wall model for immersed bodies.
 +
 +
* '''Sub-project 5: TBLE wall model for LES with pressure gradient on a simple turbomachinery geometry (M. Cizeron, N. Odier, R. Vicquelin)'''
 +
Wall modeling is often used in LES to alleviate the computational cost that would be required to resolve all the length scales up to the solid boundaries of the domain. The classical way of doing it is by using an algebraic model to provide the wall friction and heat flux, with a coupling to the LES solver at the first off-wall nodes. The wall model was designed from analyzing RANS equation with strong assumptions such as planar flow, equilibrium and no pressure gradient. These assumptions are often far from true in real applications, such as turbomachinery applications, where the use of a wall model is mandatory due to the size of the calculation. During this workshop, a wall model relying on the resolution of the Thin Boundary Layer Equations (TBLE)  was studied, which had been implemented by EM2C. The addition of a pressure gradient to these equations has been conducted and tested, at first only for the 1D wall model solver, then on a 3D turbulent channel. It remains to be tested on a diffuser configuration with a real pressure gradient to quantify the effect of the new wall model. The influence of the point considered to do the coupling between the LES and the wall model (ie. its distance to the wall) has also been tested both for the TBLE and the original algebraic model, showing that coupling farther from the wall yields better results and reduces the so-called log-layer mismatch.
 +
 +
* '''Sub-project 6: Tools for rough wall modelling (A. Barge, S. Meynet)'''
 +
Within the STREAM project framework, a roughness-resolved Large-Eddy Simulation (RRLES) database is being built. The aim of this latter is to be representative of rough channel flows, especially for additive-manufacturing heat exchangers. First RRLES have already been performed. From turbulence and rough wall stress statistics analysis of the results, a first stochastic model, which reproduces the statistical behavior of the wall stress vector, have been proposed. The modeled wall stress allows a better prediction of the pressure drop in a flat wall channel compared to the use of the mean value of the wall stress measured in RRLES alone. However, the near wall region is still mispredicted and the model is correlated in time but not in space. The aim of this ECFD5 was to develop tools to improve modelling and explore new ways. A roughness mapping tool for smooth surfaces have been implemented into YALES2 to get local surface height. This tool is based on an existing in-house surface roughness generator developed for the STREAM project. The idea is to use the map to generated space correlated fluctuations for the wall shear stress. Some bugs still remain to fully use this tool. In parallel, the modelling approach was extended to passive scalar, especially for temperature. To this end, new random tools as white noise, unit sphere random walk and Gaussian / Log-normal stochastic processes have been coded. Finally, the idea of using walls as velocity source terms emerged during this ECFD5. The principle is to mask a grid layer above the wall and to transport the rough map on this grid to estimate the roughness effects above the wall. Parametrizing and testing these tools remained to be done at the end of ECFD5.
  
 
=== User experience - J. Leparoux, SAFRAN TECH ===
 
=== User experience - J. Leparoux, SAFRAN TECH ===
  
 +
* '''Sub-project 1: Multi combustion model chemtable generator  (S. Dillon, R. Mercier)'''
  
=== Hackathon - G. Staffelbach, CERFACS ===
+
Due to the high computational cost of LES of turbulent combustion with detailed chemistry, tabulated chemistry approaches remain a popular choice within the combustion community. This popularity of tabulated chemistry leads to research into the development of novel combustion models for which a platform to test these models is required. The objective of the workshop was to therefore begin developing an easy-to-use chemistry table generator within Python. Given that there exists numerous turbulent combustion models (F-TACLES Diffusion, F-TACLES Premixed, 2PFT, etc.), a generalised multi-model code was necessary. Along with multi-model capabilities, the code was required to function with multiple solvers for the creation of the flame database (REGATH and CANTERA). With these objectives in mind, the code “TabulatEd Chemistry GenERator for Aeronautical CombusTion” (TECERACT) was created. Advancements were made in all key areas using code already developed by Safran Tech. TECERACT includes a parallel flame database generator and current developments are focused on multi-model functionality and simplified mesh management within Python. Once finished, the TECERACT code will provide a user-friendly chemistry table generator capable of producing chemistry tables for LES simulations and its multi-model structure will serve as a platform for future turbulent combustion model development/testing.
 +
 
 +
* '''Sub-project 2: Task-driven automatic run sequence (R. Mercier, J. Leparoux, M. Cailler, R. Letournel)'''
 +
 
 +
The principal objective of this project was to develop a new simulation workflow based on Task-driven approach. This approach could offer a higher flexibility to manage CFD simulations by decomposing the whole simulation on small sequences (run initialization, fuel injection, etc.)
 +
 
 +
This week showed the achievement of the first steps towards a complete task-driven simulation. (i) The whole simulation workflow was set-up (ii) Missing actions were implemented allowing to update simulation status on-the-fly (from non reactive to reactive simulation) (iii) State are now embedded from the restart solution allowing to restart from the last known state. Further work is still needed to obtain the full implementation of the AMC framework based on task-driven approach. Especially to automate the adaptation loop
 +
 
 +
* '''Sub-project 3: YALES2 Tools & Gitlab CI (J. Leparoux, A. Tstetoglou, R. Mercier)'''
 +
 
 +
In recent yers, Continuous Integration (CI) has become a good practice for software development allowing to detect integration errors quickly. In this context, running test cases with an automated execution is required to ensure the success of the CI strategy. CI is already deployed on about 60 tests cases of the YALES2 distribution.
 +
During the ECFD week, a new srategy was setup based on a Docker image allowing to manage the OS environment by controlling dependencies and library versions used. Hence, a better operability is reached to execute the CI. The CI was also extended on YALES2 tools with the creation of unit tests for python tools which should be pursued.
 +
 
 +
An other achievement of the week refers to the improvment of Y2 smart archive tool. Y2 smart archive is a useful tool which gives the user more freedom when it comes storing their data. Y2 smart archive allows for automatic smart archiving and has the capability of taking user-specified maximum number of tar files and/or the maximum tar file size. The python tool conserves the original tree structure of the folder and has the ability to automatically detect the tree depth where the tar files are to be saved. During the project, y2 smart archive was tested on a windows computer. This implementation verified the ability to archive save a whole Windows computer with automatic handling of multi-platform symlinks. In the case where files may be skipped, log files are stored for reference. Current developments of the tool include automatic background launching, which will result in a reduction of required user-input hours.
 +
 
 +
* '''Sub-project 4: Wind energy tools (E. Muller, S. Gremmo, F. Houtin-Mongrolle, B. Duboc)'''
 +
 
 +
Set-up phase of simulations involving several wind turbines is very demanding and error-prone, especially for simulations carried out with the coupled solver YALES2-BHawC*. This type of simulation can involve several dozen input files and then, an assistant tool is required.
 +
 
 +
The objectives of this projet were (i) develop a python tools to set all necessary ACTUATOR_SET objects (ii) provide basic visualization plots allowing to monitor simulations on-the-fly (by post-processing YALES2 temporals and BHawC results files).
 +
 
 +
The first step has been successfully completed by developing a python tool ''y2_actuator_sets.py''. This tool will be very useful for users to setup a YALES2 wind energy simulation with a single inputfile instead of a dozen inputfiles and with printed insights allowing to users to check inputs and errors faster than before.
 +
Some works should be pursued to improve the monitoring tool using plotly with HTML plots.
  
 +
''*BHawC : aero-servo-elastic solver used and developped at Siemens Gamesa Renewable Energy for wind turbine design and certification
 +
''
  
 
<!--
 
<!--

Latest revision as of 15:18, 6 February 2022


Description

ECFD5 workshop logo.
  • Event from 23th to 28th of January 2022
  • Location: Centre Bonséjour, Merville-Franceville, near Caen (14)
  • Two types of sessions:
    • common technical presentations: roadmaps, specific points.
    • mini-workshops. Potential workshops are listed below.
  • Free of charge
  • More than 50 participants from academics (CERFACS, CORIA, IMAG, LEGI, EM2C, UMONS, UVM, VUB, UCL, TUDelft), HPC center/experts (GENCI, AMD, CINES, CRIANN) and industry (Safran, Ariane Group, Siemens-Gamesa).
  • Objectives
    • Bring together experts in high-performance computing, applied mathematics and multi-physics CFDs
    • Identify the technological barriers of exaflopic CFD via numerical experiments
    • Identify industrial needs and challenges in high-performance computing
    • Propose action plans to add to the development roadmaps of the CFD codes

News

  • 03/11/2021: First announcement of the 5th Extreme CFD Workshop & Hackathon !

Banniere ECFD5 sponso.png

  • 13/01/2022: After discussions with the participants, the 5th Extreme CFD Workshop & Hackathon is maintained as an in-person event! It will be also possible to attend to the plenary sessions and participate remotely to the workshop.
  • 14/01/2022: The ECFD5 program is online! The plenary sessions will be announced soon!
  • 20/01/2022: The plenary sessions are now defined:
    • P1 - 24/01/2022: GPU porting challenges and quantum computing, présentation machine Adastra by G. Staffelbach (CERFACS) + Presentation of the new cluster from CINES called Adastra by C. Andrieu (CINES)
    • P2 - 25/01/2022: News, perspectives and future of GPU computing applied to CFD by A. Toure (AMD)
    • P3 - 26/01/2022: Theory, applications and perspectives of the Lattice-Boltzmann Method by P. Boivin (M2P2)
    • P4 - 27/01/2022: Concepts and notions of mesh adaptation by C. Dapogny (LJK)
  • 28/04/2022: The ECFD5 event is now finished! It was again a successful scientific week. See you next year for the ECFD6! LinkedIn post

ECFD5 PhotoGroupe.jpeg

Agenda

ECFD5 program.png

Thematics / Mini-workshops

These mini-workshops may change and cover more or less topics. This page will be adapted according to your feedback.

Projects

Hackathon - G. Staffelbach, CERFACS

  • Participants: I. d'Ast, J. Legaux, G. Staffelbach, P. Begou, G. Lartigue, V. Moureau, A. Toure, C. Laurie, S. Delamare, C. Andrieu, C. Jourdain

While hybrid CPU/GPU computing has seen growing interest in the CFD community, AMD GPU hardware is a newcomer for this application. This hackathon was the opportunity to deep dive into the AMD development environment and to prepare the arrival of AdAstra at CINES, a new super-computer which relies on AMD CPU and GPU hardware. Two flagship codes of the applied CFD community, namely YALES2 and AVBP, have been ported to the AOMP framework using ROCm 4.5 on the GRID5000 Neowise system. This machine [1] was donated by AMD to GENCI and INRIA to help in fighting the COVID pandemic and it is operated by the Grid'5000 infrastructure.

Part of the project was spent in porting the codes to the new development environment. This work posed no real issues and CPU execution was achieved fast so that we were able to focus on GPU Offloading using OpenMP. On the YALES2 side, a mini-app encompassing the typical YALES2 structure hierarchy and loop execution was extracted from the code to evaluate different porting strategies. On the AVBP side, the current OpenACC GPU offloading was translated to OpenMP focusing on the viscosity computation kernel. We learnt that the current supported standard of OpenMP in ROCm 4.5 does not allow for direct offloading of reference values inside a derived type structure but is was possible to use aliases such as pointers or flat array copies to offload. This should be solved with the support of OpenMP 5.0. Another troublesome issues, was the lack of support for offloading of array vector operations (ex : array(:) = 1.0 ) rendering the explicitation of the loops for these manadatory.

Some bugs remain and it is encouraged to use the latest compiler version when working on the porting (the release of flang 14.0.1 saved us a lot of time as we had started with 14.0.0). Offloading of the miniapp of YALES2 yielded a times 60 acceleration of the kernel whereas the offloading of the viscosity model in a full avbp simulation yielded an 7 times factor in performance when comparing on core to one GPU. These results are to be taken with a grain of salt but are really encouraging. For the next steps, a porting strategy for both codes will be implemented (depending on the OpenMP 5 support) and discussions are underway with CINES and other partners so as to offer the best experience to both code's communities on AdAstra at its release.

Combustion - K. Bioche, VUB

  • Sub-project 1: H2/air jet-in-cross-flow numerical simulations (R. Le Dortz, E. Riber, Q. Douasbin)

The use of hydrogen as an aviation fuel requires new combustion chamber design. Among strategies to prevent flame flashback and low flame residence time, the micromix injection system is further studied by ENABLEH2. This systems corresponds to a multitude of H2/air jet-in-cross-flow configurations. A 3D numerical simulation with realistic thermodynamics and kinetics is now tractable thanks to massively parralel computing. This week saw the completion of the first steps towards the establishment of a complete simulation. (I) The non-reactive air injection in the combustion chamber. (II) The cross-injection of H2 without ignition. (III) The ignition of this mixture modeled with the skeletal kinetic mechanism of Boivin (H2, H, O2, OH, O, H2O, HO2, H2O2, N2). Further work will be realised concerning mesh refinement, modelling of NOx and porting of the computation on GPU.

  • Sub-project 2: LES calculation of the MICADO test rig with multicomponent jetA1 (S. Puggelli, T. Lesaffre, E. Riber, B. Cuenot)

The EU-funded project ALTERNATE has the goal of exploring the possibility for a wider utilisation of aviation sustainable fuels. A part of the project deals with the assessment of the effect of SAFs on soot production: using the experimental information obtained at ONERA in high-pressure conditions on the MICADO test rig, the effect of Alcohol to Jet (ATJ-SPK) fuel on soot levels are assessed and compared with standard jet A1 emissions. During the project, STech and CERFACS are working jointly on the numerical modelling of soot emissions for jet-A1 and ATJ-SPK combustion in AVBP. Starting from the numerical setup under-development for jet-A1, the worshop permitted to: (I) Switch from a 2-step kinetic mechanism to a complex 29 species, 233 reacs and 15 QSS mechanism. This transition was efficiently conducted with the tool Multi Table Generator. (II) At this stage, an assessment of the effects of the flame sensor on the calculation results was carried out, indicating the consistent behaviour of a recently developped sensor w.r.t classical tools. (III) Switch towards a multicomponent formulation of jet-A1 and assessment of the effect of such advanced approach with respect to the single-component formulation previously employed. Further work will be realised to manage the stiffness of employed kinetics and to compare jet-A1 and ATJ-SPK fuels from a chemical point of view.

  • Sub-project 3: Euler-Lagrange Multigrid Simulation (T. Lesaffre, O. Vermorel, E. Riber, B. Cuenot)

In Lagrange simulations, the point-source approach is based on a ponctual approximation of the particule and requires this last to be smaller than the mesh. The very fine meshes required to represent the Eulerian phase of Euler-Lagrange two-phase flow simulations can lead to a non-validity of the point-source hypothesis. This project aimed at implementing, in the AVBP solver, the simultaneous management and coupling of several simulations. During this week, the Eulerian and Lagrangian phase were successfuly computed on two different meshes and coupled via the CWIPI library. The good behaviour of this framework was assessed on a 1D Evaporation of kerosene droplets in an air stream test case. Encouraging preliminary performance results were obtained on a 3D injection case and require further work.

  • Sub-project 4: Devolatilization modelling for biomass combustion (K. Bioche, L. Bricteux)

Biomass combustion simulations require the modelling of numerous physical phenomena: particle drying, devolatilization, gas-phase combustion, chars oxidation. Besides, the valorisation chains for biomass include fluidized bed reactors, fixed bed reactors and pulverized fuel burners. The Granular Flow Solver of YALES2 offers a good framework for the simulation of fluidized bed reactors and is functionnaly coupled with the reactive gas-phase solver of the same code. This week permitted to partically implement the modelling of devolatilization in this solver. A single-step kinetic scheme is considered for the particle mass evolution equation while the particle diameter evolves during the process. Further work is necessary to account for the thermal and mass couplings with the fluid phase.

  • Sub-project 5: Thickened-Flame LES model in a Lattice-Boltzmann Method framework (P. Boivin, S. Zhao, M. Le Boursicaud)

The TFLES framework of the hybrid Lattice-Boltzmann sover ProLB was extended to account for recent sensor methods. During this week, a smooth flame sensor based on the curvature of the norm of the advancement variable gradient was developped. Also for filtering operations, the lattice requires to access data over three neighboring layers. A precise and continuous thickening factor was obtained with such method.

  • Sub-project 6: NOx modeling applied to KIAI combustion chamber (J. Obando, P. Bénard, V. Moureau)

This project treated of the implementation of NOx modeling into simulations of the KIAI combustion chamber, experimentaly studied at CORIA lab. During this week, various NOx modeling strategies were listed. Associated kinetic mechanisms, among which analytical chemisty, were employed for 1D flame simulations in YALES2 solver. Further work include the use of such methods on the 3D computational case.

Static and dynamic mesh adaptation - G. Balarac, LEGI

  • Sub-project 1: Anisotropic mesh adaptation in YALES2 (A. Grenouilloux, A. Stock, G. Lartigue, G. Balarac, A. Froehly, C. Dapogny, J. Leparoux)

Mesh adaptation based on isotropic meshes is now a mature tool in YALES2 based on the Mmg library. This parallel mesh adaptation method is very effective for increasing the resolution in sensitive regions of the flow even when using billion-cell meshes. In many configurations, a significant gain in terms of computational cost can be expected through the use of anisotropic mesh. However, this requires to address several aspects: (i) the ability to generate anisotropic meshes via the coupling between YALES2 and Mmg, (ii) to define an anisotropic metric adapted to the simulated flow, and (iii) to assess the properties of the numerical methods (prediction/stability) of YALES2 on such meshes. This project has addressed the two first points (see sub-project 5, in Numerics for the last one). During the workshop, a full anisotropic mesh adaptation strategy has been implemented in YALES2. Anisotropic mesh adaptation needs to prescribe a metric tensor to define the ellipsoid characterizing the anisotropic tetrahedral element. First developments were then devoted to the manipulation and the control of this metric tensor (metric intersection, graduation control, aspect ratio control, etc...). Second part of the project was devoted to the determination of an anisotropic mesh based on computed fields. The Hessian-based metric definition has been implemented for this purpose. Successfull demonstrations have been conducted for 2D and 3D meshes during the workshop paving the way to upcoming applications.

Multi-phase flows - M. Cailler, SAFRAN TECH

  • Sub-project 1: Hybrid E-E/E-L two-phase flow method (M. Cailler, F. Pecquery, I. El Yamani, V. Moureau)

The High-Fidelity approach based on ACLS & DMA allows a reliable description of interface dynamics. For design exploration, low-CPU methods with controlled level of fidelity are required. An interesting approach to reduce CPU cost relies on an hybrid approach based on an Eulerian representation of the gas and a Lagrangian description for the liquid phase. Objective of the ECFD5 was to explore the capability to reconstruct the interface normal of a liquid droplet made of particles on an Eulerian grid. First, a level-set based strategy relying on Geometric Multiple Markers Projection (Janodet et al., 2022) has been first tested showing good capabilities providing that the iso-surface distance equal 0 is well captured on the mesh. An alternative strategy based on the liquid volume fraction has been tested. This information was then used to build a velocity correction that is used to transport particles and ensures regularized particle positions. This preliminary benchmark work will be pursued on a liquid jet propagation problem.

  • Sub-project 2: Jet atomization with a diffuse interface mathod (N. Odier, B. Péden, J. Carmona, P. Boivin)

A so-called diffuse interface method, using the multi-fluid formalism, coupled with a Riemann solver HLLC and a thermodynamic closure of type Noble-Abel Stiffened Gas (NASG), was implemented in AVBP during the thesis of Julien Carmona. During the ECFD5, the NASG thermodynamic was coupled with an improved HLLC solver implementation based on nodal information only, therefore leading to better performances in parallel. Moreover, the NSCBC terms expressed in the framework of the NASG thermodynamics were accurately derived. Future works include validation of the new solver capability on 1-D shock tube and isolated bubble test cases.

  • Sub-project 3: Convergence of the interface curvature computation (G. Ghigliotti, J. Carmona, G. Balarac, G. Lartigue)

The computation of interface curvature in a level-set framework is based on the divergence of the gradient of the distance to the interface. This function being computed at 2nd order, one obtains a O(0) curvature, meaning that the error does not decrease with mesh refinement. We have implemented in YALES2 a strategy proposed by Emmanuel Maître and collaborators. This approach, implemented in a finite element code FreeFEM++, is based on the regularization (filtering) of the level-set gradient and curvature. This strategy has been tested for the simple test case of a static circular interface. We used two types of filters (simple gather-scatter or bilaplacian as developed by Lola Guedot (PhD thesis 2015)) on different mesh types (split quadrilaterals, isotropic triangular mesh, unstructured triangular mesh). The results are encouraging since a O(1) convergence is obtained in all cases. Further work is needed to tune the filter properties (amplitude and size) for different spatial resolutions and levelset "narrow band" width.

  • Sub-project 4: Conservative two-fluid momentum transport (F. Pecquery, C. Merlin, M. Cailler, J. Carmona, V. Moureau)

The aim of this project was to investigate the applicability of the conservative two-fluid transport framework developed for passive scalars to the momentum conservation equation. First a complete two-fluid framework for the incompressible two-phase Navier-Stokes equations was proposed. Then the algorithms for discontinuous scalars (data extension in the other phase, reinitialization and transport) have been extended to discontinuous vectors. Moreover, some improvement of the data-structure were implemented to further generalize the framework and improve user-experience. To eventually conclude on the applicability of the two-fluid momentum approach, the proposed interfacial momentum flux model and correction step will be implemented in YALES2.

Numerics - G. Lartigue, CORIA

  • Sub-project 1 : High order schemes for distorted meshes (M. Bernard, G. Balarac, G. Lartigue)

The high order framework (HOF) based on deconvolution recently developed and implemented in Yales2 permits to increase the accuracy of spatial numerical schemes on distorted meshes. Nevertheless, while dealing with highly anisotropic meshes (aspect-ratio above 10), the conditioning of the convolution matrix gets worse, complicating its inversion.

The aim of this project was too improve the robustness of the HO3 numerical scheme while dealing with highly anisotropic meshes. The strategy was to ensure third order accuracy as far as possible, then if conditioning gets too bad, to locally decrease the solution at 2nd order, using the HO2 convolution matrix which is unconditionally invertible. Then, solution must be reconstructed with care at control volume interface to account for this mixing between HO2 and HO3. Implementation has been successfully tested on transport of a scalar field in a non-uniform flow field with aspect-ratio up to 100.

  • Sub-project 2: A coupling between YALES2 solver and a VPM method for the simulation of external flows (P. Billuart, G. Lartigue, P. Bénard)

Numerical simulations of flows past immersed objects are ubiquitous in both research and industry, spanning e.g. vehicle aero- or hydrodynamics, biological propulsion, wind energy, civil engineering, etc. Such problems are typically solved using either a (1) body-fitted grid solver, like the YALES2 finite volumes solver, or via (2) a vortex method, like the VPM (Vortex Particle Mesh) solver. Vortex methods are indeed quite appealing in that problem because they yield very low dispersion and diffusion errors which puts them among the most efficient methods to accurately advect vortical structures over long distances. In addition, their vorticity-based formulation accommodates quite naturally the enforcement of the unbounded conditions of external flows. However, vortex methods are much less suited to capture boundary layers than classical body-fitted grid solvers, e.g. finite elements, finite volumes, etc, because of their isotropic computational elements. This leads to the conclusion that one could exploit the advantages of vortex methods and body fitted-grid solvers in a coupled approach; using YALES2 for solving the near-wall part of the flow, while solving the wake using the VPM method. This project aimed to implement such a coupling using the CWIPI library. It requires two information exchanges between both solvers; (a) an interpolation of vorticity and velocity VPM fields onto the outer YALES2 boundary and, (b) the interpolation of the YALES2 velocity field onto the VPM grid. While tests regarding the MPI communicator splitting, CWIPI initializations and CWIPI interpolation initializations demonstrate a correct implementation of those, they also point out that a bug still remains in the interpolation process from VPM to YALES2.

The next steps will be: (1) keeping on identifying this bug, (2) implementing the information exchange from YALES2 to VPM and (3) synchronizing both solvers on the global communicator via MPI calls.

  • Sub-Project 3: Assessment of the YALES2 compressible flow solvers (CPS,ECS,SIC) for turbomachinery applications (P. Tene, L. Bricteux, S. Lavagnoli, Y. Béchane, P. Bénez)

High propulsive efficiency calls for Ultra-High By-Pass ratio geared turbofans. A huge fan ensuring the high dilution is rotating slowly. This fan is driven by a low-pressure turbine (LPT) rotating fast (diminishing stage count and weight). Both components are linked using a gearbox. As the fan produces 80% of the thrust, the LPT that drives it is a key component to achieve efficiency. LPT operates at transonic exit Mach numbers and low-Reynolds numbers (60-80k). Within this range of operating conditions, there is a critical shortage of aerodynamic and performance measurements.

This week we assessed the ability of the YALES2 Explicit compressible flow solver (ECS) to ensure consistent results for such applications. The first results were very promising, and proposals were made (e.g imposing total pressure as a boundary condition) to improve the development of this solver. In addition, an assessment of the different numerical schemes of this solver in terms of diffusion has also been carried out. Advances have also been made in the development of the YALES2 semi-implicit compressible solver (SIC). This solver will eventually (in the context of this application), improve the computational cost because unlike ECS, the time step here is not limited by the acoustic CFL.

  • Sub-project 4: Avoiding the acoustic timestep restriction in two-phase flow simulations (V. Boniou, J. Paris, A. Vié, T. Schmitt, C. Tenaud, Y. Béchane)

In the context of simulating compressible two-phase flows with phase change, the CFL constraint related to the acoustic part of an explicit solver, especially in the liquid phase, can be prohibitive for evaporation studies implying large timescale separation between evaporation and acoustic wave propagation. The acoustic CFL constraint can be either relaxed by considering an implicit treatment of the acoustic part or avoided by deriving the low Mach limit of the multifluid model.

During this week, the first steps of this vast work have been initiated on single-phase flows by implementing the variable-density low Mach solver and the IMEX method applied on the Euler equations in TITAN.

The IMEX method was shown to provide degraded solutions in acoustic-driven cases, compared to conventional Riemann solutions. However, in transport-dominated cases, the increase of the transport CFL leads to a better numerical precision. A 2D cylindrical shock has been simulated with an acoustic CFL of 40 to assess the ability to run multidimensional compressible computations with large time step. The method was also extented to 4eq and 2nd order spatial accuracy.

Finally, the extension of the variable density solver to two-phase flows was performed using a VOF interface capturing method, considering a dilatable gas phase along with an incompressible liquid phase. Particular attention has been drawn to preserving liquid volume by treating the VOF transport accordingly with an adapted velocity field.

  • Sub-project 5 : Numerical schemes for anisotropic meshes (R. Letournel, G. Lartigue, G. Balarac, M. Bernard)

Many efforts are devoted to the development of mesh adaptation based on anisotropic meshes in Yales2. For many configurations (e.g. shocks, mixing layers, near wall flows), a significant gain in terms of computational cost can be expected through the use of such meshes. The aim of this project was to characterize Yales2 numerical schemes on anisotropic meshes in comparison with isotropic meshes.

The strategy was at first to test different Laplacian operators on analytic functions (linear, quatratic, bump in one or two directions) in direct sense (apply the Laplacian operator) and indirect (solve the Poisson equation), to verify the order of convergence of the operators. Different meshes were considered: Isomesh, Quad2tri, MMG meshes, with different stretching (corresponding to anisotropic ratios). This exhaustive study showed that only the SIMPLEX operator remains of order 2 for the Poisson solver, while the IGNORE_SKEWNESS and PAIR_BASED operators do not converge.

In a second step, the performance of the incompressible solver on a 2D Poiseuille flow have been evaluated for increasingly stretched meshes. The coarsening in the preferential direction of the flow allows to decrease the convective time step and thus to reduce the RTTS (Return Time To Solution), while keeping the same error. These conclusions are valid for anisotropic ratios of the order of 10, beyond which the time step no longer decreases and further studies are needed to understand the limitations in these cases.

  • Sub-project 6 : Poisson solver comparison for a two-phase flow CFD code (J. C. Brändle de Motta, A. Poux, T. Ménard)

Poisson solvers known to be highly sensitive to discontinuities and represent the main computational costs of incompressible codes. Archer is a mature two-phase flow CFD code. Its historical Poisson solver is currently based on a Conjugate gradient with multigrid preconditioner (MGCG). The aim of this project was to couple Archer and the PETSc library, assess its performances and challenge the historical solver of Archer.

The coupling with PETSc has been implemented during the workshop and the comparison with MGCG has been done up to 2 thousands cores in single and two phase turbulent flow. Multiple PETSc configuration has been tested, and show that MGCG is very competitive (one order of magnitude faster than PETSc)

Turbulent flows - P. Bénard, CORIA

  • Sub-project 1: Optimization of the actuator set for several wind turbines in YALES2 (F. Houtin Mongrolle, S. Gremmo, E. Muller, B. Duboc)

The implementation of the Actuator Line Method (ALM) into the YALES2 library leads to poor performances when many wind turbine rotors are set. Indeed, each rotor object is a derived type treated sequentially by all the processors participating to the computation. With 30 turbines in a computation, the return time is increased by 70% while the arithmetic intensity appears to be low. The objective of this sub-project is to improve the computation performances of the ALM already identified: (i) Assign one MPI communicator by rotor object gathering the processors close to the turbine and set-up a master/slave processus by communicator. This will allow the simultaneous rotors computation and reduce the number of MPI exchanges. (ii) Work on the domain decomposition to limit the number of processors attributed to each turbine. This would reduce or even eliminate MPI communications. (iii) Adapt the YALES2-BHawC aero-servo-elastic coupling to make it consistent with the new ALM implementation. During the workshop, the (i)-algorithm has been implemented with additional optimizations. Tests on a 30-turbine wind farm showed a 60% improvement compared to previous performances, which is very promising. The (ii)-feature still needs to be developed but should lead to the performances objective. The tests with the coupled code YALES2-BHawC were not conclusive yet and still require some work.

  • Sub-project 2: Thermal effect in an atmospheric solver (U. Vigny, L. Voivenel, S. Zeoli, P. Benard)

Given the current environmental and energy challenges, maximising the wind farm electricity production is essential. Therefore, it becomes necessary to develop the most reliable and accurate prediction and simulation tools. Following this tenet, an atmospheric solver, which will take into account meteorological phenomena, should be developed. The preliminary work, going from bibliography study to road map was performed during the extreme cod workshop. Thus five parts have been identified: (I) The YALES2 Variable Density Solver (VDS) will be used because of the need to take into account buoyancy effect including for big density differences. (II) A wall law correction term, relative to atmospheric boundary layer will be added. (III) The actuator line method used to simulate wind turbine will be extended to VDS, modifying the velocity source term to a momentum source term. (IV) The Coriolis effects, depending on the latitude will be implemented. (V) The wall heat flux, allowing to simulate diurnal and nocturnal cycles on various terrains, is more realistic than a target wall temperature. From this work, future development are now clear and just waiting to be developed.

  • Sub-project 3: Dynamic stall correction models for Horizontal Axis Wind Turbine (S. Gremmo, E. Muller, B. Duboc, F. Houtin-Mongrolle)

The Actuator Line Method implementation into YALES2 library suffers from the lack of correction models for some 3D unsteady effects. An important one, the dynamic stall, implies an unsteady modification of the aerodynamic loads, i.e. the polars, with the angle of attack. With the standard ALM model, static 2D polars are used. Adding a dynamic stall correction model allows to have an hysteresis loop on the evolution of the forces coefficients with the angle of attack. Several models exist in the literature: Oye, BeddoesLeishman, Sheng, Risoe, ONERA... During the workshop, the Oye model was selected, because of its simplicity, and implemented. First, new ingredients necessary to the model were added in the polars look-up table generator used to describe the blades. Then, the Oye correction was integrated into the YALES2 library. Finally, the model was tested on simple configurations. It is now essential to further validate the implementation.

  • Sub-project4: Immersed Boundary solvers uniformisation into YALES2 (I. Tsetoglou, M. Cailler, P. Benez, S. Mendez, P. Benard)

A novel Conservative Lagrangian Immersed Boundary (CLIB) method [Tsetoglou et al., 2021] has been developed by CORIA & Safran Tech for wind turbines & gearbox lubrication simulations. Nevertheless, two different implementations of the same method were created that tends to diverge. This sub-project aims to develop a shared numerical kernel (CLIB) on which 2 solvers are based: CIB for aerodynamic applications (ICS + CLIB) and TPS for two-phase flows (SPS+CLIB). The newly developed kernel and solvers were tested and validated on test cases: 2D flows around static/rotating/oscillating disk, 3D flow around a rotating cylinder and 2D flows around counter-rotating ellipses. The newt steps involve the continuation of the validation of both solvers and its documentation, as well as the development of wall model for immersed bodies.

  • Sub-project 5: TBLE wall model for LES with pressure gradient on a simple turbomachinery geometry (M. Cizeron, N. Odier, R. Vicquelin)

Wall modeling is often used in LES to alleviate the computational cost that would be required to resolve all the length scales up to the solid boundaries of the domain. The classical way of doing it is by using an algebraic model to provide the wall friction and heat flux, with a coupling to the LES solver at the first off-wall nodes. The wall model was designed from analyzing RANS equation with strong assumptions such as planar flow, equilibrium and no pressure gradient. These assumptions are often far from true in real applications, such as turbomachinery applications, where the use of a wall model is mandatory due to the size of the calculation. During this workshop, a wall model relying on the resolution of the Thin Boundary Layer Equations (TBLE) was studied, which had been implemented by EM2C. The addition of a pressure gradient to these equations has been conducted and tested, at first only for the 1D wall model solver, then on a 3D turbulent channel. It remains to be tested on a diffuser configuration with a real pressure gradient to quantify the effect of the new wall model. The influence of the point considered to do the coupling between the LES and the wall model (ie. its distance to the wall) has also been tested both for the TBLE and the original algebraic model, showing that coupling farther from the wall yields better results and reduces the so-called log-layer mismatch.

  • Sub-project 6: Tools for rough wall modelling (A. Barge, S. Meynet)

Within the STREAM project framework, a roughness-resolved Large-Eddy Simulation (RRLES) database is being built. The aim of this latter is to be representative of rough channel flows, especially for additive-manufacturing heat exchangers. First RRLES have already been performed. From turbulence and rough wall stress statistics analysis of the results, a first stochastic model, which reproduces the statistical behavior of the wall stress vector, have been proposed. The modeled wall stress allows a better prediction of the pressure drop in a flat wall channel compared to the use of the mean value of the wall stress measured in RRLES alone. However, the near wall region is still mispredicted and the model is correlated in time but not in space. The aim of this ECFD5 was to develop tools to improve modelling and explore new ways. A roughness mapping tool for smooth surfaces have been implemented into YALES2 to get local surface height. This tool is based on an existing in-house surface roughness generator developed for the STREAM project. The idea is to use the map to generated space correlated fluctuations for the wall shear stress. Some bugs still remain to fully use this tool. In parallel, the modelling approach was extended to passive scalar, especially for temperature. To this end, new random tools as white noise, unit sphere random walk and Gaussian / Log-normal stochastic processes have been coded. Finally, the idea of using walls as velocity source terms emerged during this ECFD5. The principle is to mask a grid layer above the wall and to transport the rough map on this grid to estimate the roughness effects above the wall. Parametrizing and testing these tools remained to be done at the end of ECFD5.

User experience - J. Leparoux, SAFRAN TECH

  • Sub-project 1: Multi combustion model chemtable generator (S. Dillon, R. Mercier)

Due to the high computational cost of LES of turbulent combustion with detailed chemistry, tabulated chemistry approaches remain a popular choice within the combustion community. This popularity of tabulated chemistry leads to research into the development of novel combustion models for which a platform to test these models is required. The objective of the workshop was to therefore begin developing an easy-to-use chemistry table generator within Python. Given that there exists numerous turbulent combustion models (F-TACLES Diffusion, F-TACLES Premixed, 2PFT, etc.), a generalised multi-model code was necessary. Along with multi-model capabilities, the code was required to function with multiple solvers for the creation of the flame database (REGATH and CANTERA). With these objectives in mind, the code “TabulatEd Chemistry GenERator for Aeronautical CombusTion” (TECERACT) was created. Advancements were made in all key areas using code already developed by Safran Tech. TECERACT includes a parallel flame database generator and current developments are focused on multi-model functionality and simplified mesh management within Python. Once finished, the TECERACT code will provide a user-friendly chemistry table generator capable of producing chemistry tables for LES simulations and its multi-model structure will serve as a platform for future turbulent combustion model development/testing.

  • Sub-project 2: Task-driven automatic run sequence (R. Mercier, J. Leparoux, M. Cailler, R. Letournel)

The principal objective of this project was to develop a new simulation workflow based on Task-driven approach. This approach could offer a higher flexibility to manage CFD simulations by decomposing the whole simulation on small sequences (run initialization, fuel injection, etc.)

This week showed the achievement of the first steps towards a complete task-driven simulation. (i) The whole simulation workflow was set-up (ii) Missing actions were implemented allowing to update simulation status on-the-fly (from non reactive to reactive simulation) (iii) State are now embedded from the restart solution allowing to restart from the last known state. Further work is still needed to obtain the full implementation of the AMC framework based on task-driven approach. Especially to automate the adaptation loop

  • Sub-project 3: YALES2 Tools & Gitlab CI (J. Leparoux, A. Tstetoglou, R. Mercier)

In recent yers, Continuous Integration (CI) has become a good practice for software development allowing to detect integration errors quickly. In this context, running test cases with an automated execution is required to ensure the success of the CI strategy. CI is already deployed on about 60 tests cases of the YALES2 distribution. During the ECFD week, a new srategy was setup based on a Docker image allowing to manage the OS environment by controlling dependencies and library versions used. Hence, a better operability is reached to execute the CI. The CI was also extended on YALES2 tools with the creation of unit tests for python tools which should be pursued.

An other achievement of the week refers to the improvment of Y2 smart archive tool. Y2 smart archive is a useful tool which gives the user more freedom when it comes storing their data. Y2 smart archive allows for automatic smart archiving and has the capability of taking user-specified maximum number of tar files and/or the maximum tar file size. The python tool conserves the original tree structure of the folder and has the ability to automatically detect the tree depth where the tar files are to be saved. During the project, y2 smart archive was tested on a windows computer. This implementation verified the ability to archive save a whole Windows computer with automatic handling of multi-platform symlinks. In the case where files may be skipped, log files are stored for reference. Current developments of the tool include automatic background launching, which will result in a reduction of required user-input hours.

  • Sub-project 4: Wind energy tools (E. Muller, S. Gremmo, F. Houtin-Mongrolle, B. Duboc)

Set-up phase of simulations involving several wind turbines is very demanding and error-prone, especially for simulations carried out with the coupled solver YALES2-BHawC*. This type of simulation can involve several dozen input files and then, an assistant tool is required.

The objectives of this projet were (i) develop a python tools to set all necessary ACTUATOR_SET objects (ii) provide basic visualization plots allowing to monitor simulations on-the-fly (by post-processing YALES2 temporals and BHawC results files).

The first step has been successfully completed by developing a python tool y2_actuator_sets.py. This tool will be very useful for users to setup a YALES2 wind energy simulation with a single inputfile instead of a dozen inputfiles and with printed insights allowing to users to check inputs and errors faster than before. Some works should be pursued to improve the monitoring tool using plotly with HTML plots.

*BHawC : aero-servo-elastic solver used and developped at Siemens Gamesa Renewable Energy for wind turbine design and certification