[GSoC] Feature: LU-SGS preconditioner to GPU

[ddg93]

This PR ports the LU-SGS preconditioner to GPU

Proposed Changes

This PR introduces a build phase for the LU-SGS preconditioner where the inverse of the diagonal D is calculated to facilitate the compute phase. Then, forward and backward sweeps are implemented through custom CUDA kernels matching the logic of the CPU section (except for the inverse of D, which is directly multiplied in the GPU logic). These kernels are managed by the compute method on GPU which maintains the MPI communications.

Related Work

This PR follows the discussion of #2843 . Kernel level dependencies are managed as for the ILU preconditioner proposed in #2858 .

Validation

CPU and GPU implementations of the LU-SGS preconditioner are compared for validation of this PR on the rae2822 test-case. After one iteration, the first, preconditioned Krylov vector is compared finding matching results down to machine precision. Residuals are compared after 100 iterations finding matching results.

TO-DO

• complete with the quantized_mode option path;
• run benchmarks on modern GPUs. Current evaluations on a P620 are inconclusive;
• profiling of the custom kernels;

PR Checklist

Put an X by all that apply. You can fill this out after submitting the PR. If you have any questions, don't hesitate to ask! We want to help. These are a guide for you to know what the reviewers will be looking for in your contribution.

• I am submitting my contribution to the develop branch.
• My contribution generates no new compiler warnings (try with --warnlevel=3 when using meson).
• My contribution is commented and consistent with SU2 style (https://su2code.github.io/docs_v7/Style-Guide/).
• I used the pre-commit hook to prevent dirty commits and used pre-commit run --all to format old commits.
• I have added a test case that demonstrates my contribution, if necessary.
• I have updated appropriate documentation (Tutorials, Docs Page, config_template.cpp), if necessary.

[pcarruscag]

Oh and add yourself to AUTHORS.md please

[ddg93]

The pointed issues have been addressed for the GPU LU-SGS. Also, I've added the Quantized mode through a bool runtime control passed to the kernels.

I've performed some tests on my CPU with the RAE2822 test-case:

• serial execution: CPU and GPU residuals perfectly match;
• OPENMP, 4 threads: CPU and GPU residuals match down to machine precision;
• MPI, 2 and 4 tasks: residuals diverge rapidly after few iterations, but remain comparable.

I am wondering if this divergence between CPU and GPU when running with multiple MPI tasks is expected?

To be sure, I run the same test with the ILU preconditioner and the version of the HEAD of the develop branch: matched residuals in serial and 4 OPENMP threads, non matching residuals when running with 4 MPI threads. Moreover, the GPU residuals drop and become positive. For this reason, I though that the issue might not be in the modifications introduced in this PR and pushed.

[pcarruscag]

Move to the private part of the class members

[pcarruscag]

Is it worth using the graph when the vectors are always changing? Does that only happen with FGMRES?

[pcarruscag]

Looks good, yes MPI is expected not to work, the communications are only using the CPU values, we would have to download and reupload vectors, which would be inefficient, so the best bet for now is OpenMP