VASP 6.2.0 GPU Hybrid Functional Tests Fail
Posted: Fri Mar 05, 2021 7:04 pm
Hi VASP team,
I successfully installed VASP 6.2.0 using OpenMPI a few weeks ago. Now, I'd like to install the OpenACC GPU port for VASP. I was able to set up NVIDIA SDK 20.9 and use my cluster's copy of Intel 17.0.3 MPK to make the VASP binaries. I edited the "makefile.include.linux_nv_acc+omp+mkl" file listed on the VASP wiki to the particulars of my system. I also had to manually prepend the NVIDIA SDK .../20.9/compilers/lib and .../20.9/compilers/extras/qd/lib directories to the LD_LIBRARY_PATH environment variable in order to successfully make the binaries.
Running the testsuite shows that all tests succeed except the Hybrid functional tests. I found that the Hybrid functional tests generally fail the force calculations. The "TOTAL-FORCE" section at the end of the OUTCAR shows "NaN" for each entry. Interestingly, the Electron-Ion, Ewald-Force, and Non-Local-Force entries are all correct. Digging into the force.F source code shows that the HARFOR, PARFOR, FORHF, or TAUFOR variables could be causing the issue. The TAUFOR variable is used for MetaGGA functionals, so that's likely not it. There must be some function call to calculate one of those variables which is throwing an error.
Some of the tests also fail the frequency calculations. Those don't show "NaN" values, but are instead just incorrect. I'm not sure what's going wrong there.
I tried making the VASP binaries with the "makefile.include.linux_nv_acc" file given in the .tar.gz package, but that had the same errors. It also showed "DSYEV" errors. These did not appear in my "makefile.include.linux_nv_acc+omp+mkl" tests.
Perhaps there's something wrong with my LAPACK or scaLAPACK packages? Maybe something is going wrong with the NVIDIA linear algebra packages?
Please let me know if you have any ideas.
Best,
James
I successfully installed VASP 6.2.0 using OpenMPI a few weeks ago. Now, I'd like to install the OpenACC GPU port for VASP. I was able to set up NVIDIA SDK 20.9 and use my cluster's copy of Intel 17.0.3 MPK to make the VASP binaries. I edited the "makefile.include.linux_nv_acc+omp+mkl" file listed on the VASP wiki to the particulars of my system. I also had to manually prepend the NVIDIA SDK .../20.9/compilers/lib and .../20.9/compilers/extras/qd/lib directories to the LD_LIBRARY_PATH environment variable in order to successfully make the binaries.
Running the testsuite shows that all tests succeed except the Hybrid functional tests. I found that the Hybrid functional tests generally fail the force calculations. The "TOTAL-FORCE" section at the end of the OUTCAR shows "NaN" for each entry. Interestingly, the Electron-Ion, Ewald-Force, and Non-Local-Force entries are all correct. Digging into the force.F source code shows that the HARFOR, PARFOR, FORHF, or TAUFOR variables could be causing the issue. The TAUFOR variable is used for MetaGGA functionals, so that's likely not it. There must be some function call to calculate one of those variables which is throwing an error.
Some of the tests also fail the frequency calculations. Those don't show "NaN" values, but are instead just incorrect. I'm not sure what's going wrong there.
I tried making the VASP binaries with the "makefile.include.linux_nv_acc" file given in the .tar.gz package, but that had the same errors. It also showed "DSYEV" errors. These did not appear in my "makefile.include.linux_nv_acc+omp+mkl" tests.
Perhaps there's something wrong with my LAPACK or scaLAPACK packages? Maybe something is going wrong with the NVIDIA linear algebra packages?
Please let me know if you have any ideas.
Best,
James