Project-Ananke
Project Anake is a on-onging CFD project, a highly efficient Magnetohydrodynamics (MHD) direct fluid GPU based code implemented on high level programming language julia. It is inspired by Athena++/MHDFLows.jl/FourierFlows.jl. Its goals is to achieve a native GPU based, highly parallel, and efficient MHD simulation code using Finite Volume Method(FVM).
At the end of this proejct, we aim to provide a prototype MHD code, which will support the following features:
- 1D/2D/3D uniform Grid simulation on cartesian coordinate system
- Hydrodynamics/Magnetohydrodynamics Equation Solver
- Support Isothermal/Adiabatic equation of state.
Current State of development:
Version : v.0.1.2
Phase I : Finished & under testing
Phase II : Under development of MPI boundary value exchange module
To achieve those features, we split our development into three phases:
I) Simple GPU HD simulation with HLLC/HLLE Solver supporting uniform cartesian coordinate
- Data structure & Problem construction
- Recontruction construction (DC(1)/ PLM(2)/ PPM(3) )
- Solver Construction ( HLLC for addiabatic/ HLLE for isothermal)
- Basic Integrator Construction (second-order accurate van Leer predictor-corrector scheme)
- Boundary value problem constructor (periodic/outflow/reflective)
II) MPI implementation & Curvilinear coordinate Support
- MPI Meshblock distribution
- MPI Boundary value exchange function
- Problem diagnotics for global statistics after each iteration (e.g. Total Energy)
- Global FFT contructor (with CuFFTMp)
- Support for cylindrical/ spherical coordinates (optional)
III) B-field implementation
- Reconstruction method with B-field
- HLLD Solvers
- Div B Free algorthm (CT)
Compatiblility :
Support Julia v1.8.3 with CUDA compatible GPU
Performance Evaluation:
Ananke is still under development and lack of optimzation. User should expect the perforamnce could change from time to time. As for now, Ananke is mainly memory-bound. So the performance decline moderately from swtiching between Float32/Float64 calulation. Nonetheless, as the current test is based on high-end gaming card with nerfed VRAM/FP64 Core configuration, one could expect better performance for Float64 calculation for data center GPU.
3090 24GB with 199 iterations after warm up process
| Method (M Zones Update /s ) | |||
|---|---|---|---|
| VL2 + PLM + HLLC(HD/Float32) | 86.278 | 192.651 | 242.94 |
| VL2 + PLM + HLLC(HD/Float64) | 60.918 | 157.583 | 186.07 |
| VL2 + PPM + HLLC(HD/Float32) | 74.528 | 130.43 | 155.41 |
| VL2 + PPM + HLLC(HD/Float64) | 60.996 | 98.529 | 109.78 |
The performance comparsion to Athena++ (Table 3 in Stone et al. 2020)
2× Skylake-SP Gold 614 (Total 40 Cores)
| Method (M Zones Update /s ) | about |
|---|---|
| VL2 + PLM + HLLC(HD/Float64) | 84.769 |
| VL2 + PPM + HLLC(HD/Float64) | 49.759 |
For breakdown of the runtime (i.e. VL2 + PPM + HLLC (HD/Float32) in below), as expected, Ananke spend most of the time in recontruction state & Flux construction step.
────────────────────────────────────────────────────────────────────────────────
Time Allocations
─────────────────────── ────────────────────────
Tot / % measured: 91.9s / 79.3% 91.7MiB / 93.8%
Section ncalls time %tot avg alloc %tot avg
────────────────────────────────────────────────────────────────────────────────
Time Stepper 199 71.0s 97.4% 357ms 77.2MiB 89.8% 397KiB
Flux intregation 398 67.0s 92.0% 168ms 34.7MiB 40.3% 89.2KiB
Reconstruct dir x 398 14.0s 19.3% 35.3ms 4.15MiB 4.8% 10.7KiB
Reconstruct dir y 398 13.6s 18.6% 34.0ms 4.15MiB 4.8% 10.7KiB
Reconstruct dir z 398 13.5s 18.6% 34.0ms 4.15MiB 4.8% 10.7KiB
Solver dir y 398 7.57s 10.4% 19.0ms 3.57MiB 4.1% 9.18KiB
Solver dir z 398 7.56s 10.4% 19.0ms 3.57MiB 4.1% 9.18KiB
Solver dir x 398 7.54s 10.3% 18.9ms 3.58MiB 4.2% 9.20KiB
Adding ∂F∂x 398 3.14s 4.3% 7.88ms 6.38MiB 7.4% 16.4KiB
Cons to Prims 398 2.74s 3.8% 6.89ms 15.9MiB 18.5% 41.0KiB
Boundary Exchange 398 1.18s 1.6% 2.96ms 26.5MiB 30.8% 68.1KiB
CFL 199 1.91s 2.6% 9.60ms 8.82MiB 10.2% 45.4KiB
User defined function 199 182μs 0.0% 915ns 0.00B 0.0% 0.00B
────────────────────────────────────────────────────────────────────────────────
Example
The user interface of the Ananke is inherited from MHDFlows.jl and they share most of the workflow.
Few Examples were set to illustrate the workflow of performing 1D/2D/3D simulation and its visualization.