Study Results
An algorithm for efficiently handling the tracking of a large number of individual (“Lagrangian”) particles has been implemented in the Julia programming language with an interface to OpenFOAM, that is adaptable for other solvers for the Eulerian phase. Three solvers have been developed based on original OpenFOAM solvers. Two of them, one for incompressible turbulent flows, the other for compressible, turbulent flows with heat transfer and water vapour transport, include the SCALE-TRACK algorithm. The third with OpenFOAM’s original particle tracking serves as a baseline model for comparison. The study has set up a collection of test cases to be able to validate the correct functioning of SCALE-TRACK. Scaling tests revealed very good weak scaling, where the size of a problem per computing node is held constant, and strong scaling, where the total size of the problem is held constant, also showed good efficiency up to 16 nodes. A webinar on SCALE-TRACK’s principle and its application for a cloud chamber was well received by scientists from different fields, ranging from engineering to cloud research.
For solving the governing partial differential equations of the Eulerian phase (also called continuous or fluid phase), the open source finite volume software OpenFOAM v2406 is used as a base (www.openfoam.com). Its implemented particle tracking also serves as baseline model for validating the newly implemented algorithm. That new algorithm is implemented using the Julia programming language (version 1.10.8, julialang.org), employing its CUDA and MPI capabilities. The code can be found under https://github.com/Wikki-GmbH/SCALE-TRACK.
Regarding hardware, we used a local workstation (dual AMD EPYC 9684X and Nvidia RTX 6000 ADA), mainly for testing the code, and the HPC cluster Mare Nostrum 5 with its accelerated partitions on up to 32 nodes, with each node featuring two Intel Sapphire Rapids 8460Y CPUs (40 cores at 2.3 GHz per CPU) and four Nvidia H100 GPUs with 64 GB of internal memory each.
Benefits
The SCALE-TRACK algorithm developed within this study is beneficial for simulating dispersed flow phenomena in nature and technology, where the dispersed phase consists of a large number of individual gaseous, liquid, or solid particles. A prominent example in the natural sciences is clouds, which consist of a large number of small droplets. Previously, the amount of droplets reasonably tractable with Euler-Lagrange simulations was in the order of a billion. With the new algorithm, we have demonstrated excellent scalability up to 256 billion droplets, enabling high-fidelity simulations of cloud chambers and (at least parts of) real clouds. Ultimately, this could pave the way for improving weather and climate models by deriving better parameterisations from the Euler-Lagrange simulations.
In industry and technology, notable applications include internal combustion engines, which spray a large number of fuel droplets during injection, and spray drying, as used in the chemical, pharmaceutical, and food industries. All of these processes could be studied with SCALE-TRACK, providing new insights and revealing opportunities for improvisation, ultimately leading to more efficient processes and thus saving resources.
Partners
| Wikki GmbH, Wernigerode, Germany is an SME engineering company, specialized in innovative numerical simulation techniques, especially for CFD and OpenFOAM. Within SCALE-TRACK, they served as HPC experts and for algorithm development and programming. Involved in this project were Dr. Henrik Rusche and Dr. Sergey Lesnik. |
| The Leibniz Institute for Tropospheric Research (TROPOS) TROPOS is a research institute, specialized in tropospheric aerosol and cloud research, including field and lab studies, as well as numerical simulations. They provided test and validation cases and the proof of concept. Involved in this project was Dr. Silvio Schmalfuß. |
Contact
Name: Silvio Schmalfuß
Institution: Leibniz-Insitute for Tropospheric Research, TROPOS
Email Address: schmalfuss@tropos.de