VASP 6.1.1 with Intel MKL on AMD Rome: problem with testsuite
Moderators: Global Moderator, Moderator
-
- Newbie
- Posts: 7
- Joined: Tue Jul 22, 2014 9:40 am
- License Nr.: 5-568
VASP 6.1.1 with Intel MKL on AMD Rome: problem with testsuite
We built successfully VASP 6.1.1 on Broadwell and AMD Rome.
On Broadwell, we used Intel 2018 en 2019 compilers. All tests from the testsuite pass.
On AMD Rome, it’s a different story. We used Intel 2019 and Intel 2020 compilers.
The only difference between makefile.include on Broadwell and Rome: we replaced -xHOST by -march=core-avx2
The output of the first test already shows at the beginning:
WARNING: Sub-Space-Matrix is not hermitian in DAV 1
finally resulting in
BRMIX: very serious problems
the old and the new charge density differ
We also used the undocumented variable MKL_DEBUG_CPU_TYPE=5.
Do you have any idea how to fix this problem? Or how to run the testsuite successfully on AMD Rome?
On Broadwell, we used Intel 2018 en 2019 compilers. All tests from the testsuite pass.
On AMD Rome, it’s a different story. We used Intel 2019 and Intel 2020 compilers.
The only difference between makefile.include on Broadwell and Rome: we replaced -xHOST by -march=core-avx2
The output of the first test already shows at the beginning:
WARNING: Sub-Space-Matrix is not hermitian in DAV 1
finally resulting in
BRMIX: very serious problems
the old and the new charge density differ
We also used the undocumented variable MKL_DEBUG_CPU_TYPE=5.
Do you have any idea how to fix this problem? Or how to run the testsuite successfully on AMD Rome?
-
- Administrator
- Posts: 282
- Joined: Mon Sep 24, 2018 9:39 am
Re: VASP 6.1.1 with Intel MKL on AMD Rome: problem with testsuite
Hello,
can you upload the testsuite.log, are the errors large?
We don't have access to an AMD chip right now (might change in future).
Have you tried linking to FFTW + OpenBLAS?
This might result in an almost as fast alternative compared to MKL.
can you upload the testsuite.log, are the errors large?
We don't have access to an AMD chip right now (might change in future).
Have you tried linking to FFTW + OpenBLAS?
This might result in an almost as fast alternative compared to MKL.
-
- Newbie
- Posts: 7
- Joined: Tue Jul 22, 2014 9:40 am
- License Nr.: 5-568
Re: VASP 6.1.1 with Intel MKL on AMD Rome: problem with testsuite
hi,
Thanks for considering this.
I uploaded the testsuite log to dropbox because it is quite large. Here is the link: https://www.dropbox.com/s/4jl0u2058zi9z ... og.gz?dl=0
Thanks for considering this.
I uploaded the testsuite log to dropbox because it is quite large. Here is the link: https://www.dropbox.com/s/4jl0u2058zi9z ... og.gz?dl=0
-
- Administrator
- Posts: 282
- Joined: Mon Sep 24, 2018 9:39 am
Re: VASP 6.1.1 with Intel MKL on AMD Rome: problem with testsuite
It seems your vasp binary is not properly compiled.
I suspect the reason is that you use Intel compilers on AMD hardware.
Have you considered using the gfortran (for instance gfortran-7.5.0) in combination with the MKL?
There are also other toolchains for vasp-6.1.0 we have successfully tested (so far only on Intel chips unfortunately).
I suspect the reason is that you use Intel compilers on AMD hardware.
Have you considered using the gfortran (for instance gfortran-7.5.0) in combination with the MKL?
There are also other toolchains for vasp-6.1.0 we have successfully tested (so far only on Intel chips unfortunately).
-
- Newbie
- Posts: 7
- Joined: Tue Jul 22, 2014 9:40 am
- License Nr.: 5-568
Re: VASP 6.1.1 with Intel MKL on AMD Rome: problem with testsuite
Could you be a bit more specific? What makes you conclude that VASP isn't properly compiled?
-
- Administrator
- Posts: 282
- Joined: Mon Sep 24, 2018 9:39 am
Re: VASP 6.1.1 with Intel MKL on AMD Rome: problem with testsuite
Please mind, using Intel MKL on AMD chips is "experimental" and most probably officially not supported by Intel.
That been said, there is no guarantee that the instruction set on AMD is fully compatible with the one chosen by MKL when switching on DEBUG mode.
This is what I have ment by not properly compiled. Most probably Intel MKL supports only Intel chips officially.
Alternatively, you may investigate if reducing optimization (getting rid of the avx2 support for instance in your makefile.include) would provide a working binary.
Unfortunately, we don't have AMD hardware available and thus do not have the ability to investigate your problem in detail.
If you find a solution to your problem, please post it on this thread. The community and the VASP team would appreciate that.
That been said, there is no guarantee that the instruction set on AMD is fully compatible with the one chosen by MKL when switching on DEBUG mode.
This is what I have ment by not properly compiled. Most probably Intel MKL supports only Intel chips officially.
Alternatively, you may investigate if reducing optimization (getting rid of the avx2 support for instance in your makefile.include) would provide a working binary.
Unfortunately, we don't have AMD hardware available and thus do not have the ability to investigate your problem in detail.
If you find a solution to your problem, please post it on this thread. The community and the VASP team would appreciate that.
-
- Newbie
- Posts: 2
- Joined: Mon Sep 07, 2020 9:01 am
Re: VASP 6.1.1 with Intel MKL on AMD Rome: problem with testsuite
Hi,
you might want to try unsetting MKL_DEBUG_CPU_TYPE and if that solves that problem, try to set it again but also set MKL_CBWR=AUTO, at least I know of one bug which is triggered without setting MKL_CBWR.
Best,
Tobias
you might want to try unsetting MKL_DEBUG_CPU_TYPE and if that solves that problem, try to set it again but also set MKL_CBWR=AUTO, at least I know of one bug which is triggered without setting MKL_CBWR.
Best,
Tobias