Compiling VASP 6.2.1 with IntelOneAPI 2021.3.0 - Failed tests

[frank_niessen]

Hi,

I have successfully compiled VASP 6.2.1 on Ubuntu 20.04.3 LTS with IntelOneAPI 2021.3.0 using the default

Code: Select all

makefile.include.linux_intel

makefile.include. My CPU is an Intel Core i9-10920X.

The following tests fail:

andersen_nve_constrain_fixed andersen_nve_constrain_fixed_RPR andersen_nvt_fixed bulk_InP_SOC_G0W0_sym bulk_InP_SOC_G0W0_sym_RPR bulk_SiO2_LOPTICS bulk_SiO2_LOPTICS_RPR bulk_SiO2_LPEAD bulk_SiO2_LPEAD_RPR SiC8_GW0R Tl_x Tl_x_RPR Tl_y Tl_y_RPR Tl_z Tl_z_RPR

I would be grateful for support to get this issue fixed.

Best wishes
Frank

[andreas.singraber]

Hello Frank,

thank you for reporting this! I just ran the tests which failed in your case with a similar setup, i.e. also Intel OneAPI 2021.3 with the default "makefile.include.linux_intel" BUT with an older CPU (Intel Xeon E5-2699 v4) and on a different operating system (CentOS 8 ). In my case all the tests passed... so maybe it's the combination of OneAPI 2021.3 and your newer hardware... can you please attach the log file you can find in

Code: Select all

testsuite/testsuite.log

so I can have a closer look? Maybe you can add also one of the resulting OUTCAR files, such as:

Code: Select all

testsuite/tests/andersen_nve_constrain_fixed/OUTCAR

for further inspection.

Thank you!

Best,

Andreas Singraber

[frank_niessen]

Thank for looking into this, find the files attached.

[andreas.singraber]

Thank you for the files, I'm looking into this...

I have a very strong suspicion that this may be related to: forum/viewtopic.php?f=2&t=18049

I guess you have not changed the "makefile.include.linux_intel", specifically this line?

Code: Select all

FFLAGS     = -assume byterecl -w -xHOST

According to the above forum thread you could try to replace it with

Code: Select all

FFLAGS     = -assume byterecl -w -march=core-avx2

to restrict yourself to AVX-2 (don't use AVX-512) instruction sets. Or, alternatively,

Code: Select all

FFLAGS     = -assume byterecl -w -xHOST -fp-model precise

could help to avoid numerical errors. Could you try whether any of this options would resolve the issues?

For testing, you can easily run ONLY the previously failed tests by setting

Code: Select all

export VASP_TESTSUITE_TESTS="andersen_nve_constrain_fixed andersen_nve_constrain_fixed_RPR andersen_nvt_fixed bulk_InP_SOC_G0W0_sym bulk_InP_SOC_G0W0_sym_RPR bulk_SiO2_LOPTICS bulk_SiO2_LOPTICS_RPR bulk_SiO2_LPEAD bulk_SiO2_LPEAD_RPR SiC8_GW0R Tl_x Tl_x_RPR Tl_y Tl_y_RPR Tl_z Tl_z_RPR"

before repeating the tests with

Code: Select all

make test

Thank you!

[andreas.singraber]

Hello again!

I looked through the testsuite log file and found that most errors are due to segmentation faults. Can you check whether

Code: Select all

ulimit -s

returns a number? You need to check this directly on the machine where you let the tests run (not a login node of a cluster). If it's a number please "unlimit" your stack size limit by executing

Code: Select all

ulimit -s unlimited

Then try to rerun the tests and see if as many fail as before. Note, that this setting is only valid until you close your terminal. To make it permanent you have to add it to your "~/.bashrc" (or similar) file.

The tests

Code: Select all

andersen_nve_constrain_fixed andersen_nve_constrain_fixed_RPR andersen_nvt_fixed

failed due to the kinetic energy being different in the first time step. Maybe that's related to the numerical issues with AVX-512 as discussed before.

Let's see how your test results look after all these changes...

Best,
Andreas Singraber

[frank_niessen]

Hi Andreas,

thank you for your excellent support, I did both things, using the line

Code: Select all

FFLAGS     = -assume byterecl -w -march=core-avx2

and changing

Code: Select all

ulimit -s

to unlimited and now I run smoothly through all test.

Best wishes
Frank

[andreas.singraber]

Hey Frank,

great, glad I could help! I will try to find out whether "-march=core-avx2" comes with reduced performance, although I doubt it. In case there is some concern I will report back once again.

Also, please always set "ulimit -s unlimited" whenever you run a VASP calculation, not just for the tests. With the limited stack size (which is the default on many Linux distros) people often encounter mysterious segmentation faults (including myself...). Spare yourself the headache and set it directly in your "~/.bashrc" .

All the best,
Andreas

[paulfons]

After compiling vasp.6.2.1 using the Intel 2021.3.0 compiler on my cluster and found that as reported in the past that the following tests

andersen_nve_constrain_fixed andersen_nve_constrain_fixed_MDALGO=11 andersen_nve_constrain_fixed_MDALGO=11_RPR andersen_nve_constrain_fixed_RPR andersen_nve_fixed andersen_nve_fixed_MDALGO=11 andersen_nve_fixed_MDALGO=11_RPR andersen_nve_fixed_RPR andersen_nvt_fixed andersen_nvt_fixed_MDALGO=11 andersen_nvt_fixed_MDALGO=11_RPR andersen_nvt_fixed_RPR

failed due to problems with the AVX512 instruction. It was suggested to change the FFLAGS in another post to avoid the problem, but I wanted to pass along another solution that I found via a Google search. The search found a Chinese language site (https://www.bilibili.com/read/cv1340648 ... icleDetail) that suggested the following patch:

diff --git a/src/mymath.F b/src/mymath.F
index f46d930b..919fda62 100644
--- a/src/mymath.F
+++ b/src/mymath.F
@@ -1449,9 +1449,7 @@
Ltxyz=1
DO i=1,T_INFO%NIONS
DO j=1,3
- IF (.NOT. T_INFO%LSFOR(j,i)) THEN
- Ltxyz(j)=0
- ENDIF
+ IF (.NOT. T_INFO%LSFOR(j,i) .AND. Ltxyz(j)==1) Ltxyz(j)=0
ENDDO
ENDDO

[andreas.singraber]

Hello!

Thank you, this patch has already been posted in this forum here and will be also included in the next release version of VASP.

Best,
Andreas Singraber

[andreas.singraber]

Just an update for clarification:

Further testing showed that adding

Code: Select all

-fp-model precise

to the FFLAGS does NOT solve the problem, only

Code: Select all

-march=core-avx2

instead of -xHOST, or alternatively, applying the mentioned patch does!