Enabling Unified Virtual Addressing with VASP 6.4

[sergio_tosoni1]

We installed VASP 6.4.2 on a machine equipped with two AMD Epyc cpus and an NVIDIA A100 80GB GPU. The code runs fairly well on the GPU. However, we noticed that it relies only the GPU's dedicated memory (80 GB) and any test case with a higher memory request crashes. We tried to enable the unified virtual addressing strategy to make both the GPU memory and the system memory available to VASP, but up to now without success.
Is there any specific requirement in terms of CUDA drivers or flags during the compilation to be aware of?
We are using CUDA 12.3 on Linux Centos 7.
Thank you,

[jonathan_lahnsteiner2]

Dear sergio_tosoni1,

Currently this is not possible with vasp. There were no tests done about unified virtual memory
addressing and therefore there is no knowledge yet about adressing issues in this direction.

All the best Jonathan

[jonathan_schmidt]

Hi, are there any news on this?
With the new ALP system in switzerland with NVLink-C2C on the GH200 chips this would be really useful.
best,
Jonathan

[jonathan_lahnsteiner2]

Dear Jonathan and Sergio,

We have not done any testing of VASP with unified memory on GH200 ourselves. But if you have the kernel version 6.3+ you can try and build VASP with "-gpu=unified" which requires Heterogeneous Memory Management (HMM) enabled and in principle it should work.
So just add the line -gpu=unified to your makefile.include and recompile the whole vasp code. To check if HMM was enabled on your system I would recommend you to look at this link and follow the steps there.

Please note that even if VASP is running we can not give you any guarantee that the program will work as expected. Therefore I recommend to carefully test the code with some examples. It would be great if you could share your experience with us after trying.

All the Best

Jonathan

[jonathan_schmidt]

Thank you very much for the reply and instructions. I am a bit too busy now, but I will get back to you on this once I managed to try it.