Performance evaluation of GPU-accelerated OpenFOAM simulations for industrial CFD

Abstract

The Upstream CFD (UCFD) contributions to the EXASIM project focused on the assessment of technologies to enable faster and more cost-effective simulations employing GPU accelerators. This was pushed forwards by two overarching work packages:

WP2: Development of a high-fidelity HPC benchmark suite WP5: Application of the developed techniques to industrial wind energy cases

A comprehensive and reproducible benchmark suite was designed to cover a range of geometrical and physical complexities spanning canonical academic test cases to full-scale industrial simulations. The test cases furthermore covered different application areas e.g. automotive aerodynamics and wind energy, to derive more generalizable findings. Central to the suite was the OpenFOAM Benchmark Runner (OBR) developed together with our TUM partner around G. Olenik , a toolchain enabling automated, scalable, and version-controlled parameter studies on diverse HPC systems. The benchmarking campaign centred on the low risk, low reward hybrid CPU-GPU approach, where only the linear solution step is offloaded to the external solver library Ginkgo via OpenFOAM-Ginkgo Layer (OGL). The tests assessed solver performance, energy efficiency, and cost-performance trade-offs under realistic boundary conditions and meshes. The two key performance metrics are the throughput, represented as finite volume operations per second (FVOPS) and the device efficiency, which is the throughput per device (FVOPS/device). These metrics were used for further detailed analysis and comparison in terms of scalability, time-to-solution, cost-to-solution, and energy-to-solution. Complementary validation studies for cases with reference data ensured methodological soundness. The application cases demonstrated the applicability of the hybrid approach even for large and complex cases. The outcome of the EXASIM study with the tested OGL version is that this hybrid GPU-offloading strategy for OpenFOAM has no or only limited advantages under very specific conditions e.g. cases dominated by high pressure solving effort. Additional comparisons with fully GPU-native codes highlighted the performance ceiling of OGL and clearly indicated that fully GPU resident codes need to be pursued in future work.