VASP-5.4.4pl2 with AMD Rome/AOCC/Flang?

[bill_broadley]

Greetings,

Anyone have their build notes for a vasp build Optimized for the new AMD Rome CPUs?

Apparently there's a surprisingly large jump in floating point performance on short vectors in the Zen1/Naples to Zen2/Rome upgrade, something that VASP users heavily.

I have an Intel + Intel MKL vasp build working, but apparently Intel cripples the performance by ignoring the CPU capability bits and disables the AVX2-256 support. So I'm hoping I can get VASP built with Scalapack and OpenBLAS compiled natively and avoiding the Intel compiler + MKL completely.

AMD has a clang/LLVM compiler called AOCC (currently in version 2.1) that supports C, C++, and Fortran. I was able to build OpenMPI, OpenBLAS, FFTW, and Scalapack with no problems. The VASP build got about half way done (250 .o files built) and I get:

Code: Select all

mpif90 -ffree-form -ffree-line-length-none -w -O2 -I/share/apps/amd-fftw-aocc-2.1/include -c varpro.f90
F90-S-0081-Illegal selector - KIND value must be non-negative  (varpro.f90: 83)
F90-S-0081-Illegal selector - KIND value must be non-negative  (varpro.f90: 84)
  0 inform,   0 warnings,   2 severes, 0 fatal for dot_product_mpr
F90-S-0081-Illegal selector - KIND value must be non-negative  (varpro.f90: 105)
F90-S-0081-Illegal selector - KIND value must be non-negative  (varpro.f90: 106)
  0 inform,   0 warnings,   2 severes, 0 fatal for sum_mpr
F90-S-0081-Illegal selector - KIND value must be non-negative  (varpro.f90: 119)
  0 inform,   0 warnings,   1 severes, 0 fatal for toreal
F90-S-0081-Illegal selector - KIND value must be non-negative  (varpro.f90: 426)
F90-S-0081-Illegal selector - KIND value must be non-negative  (varpro.f90: 427)
F90-S-0081-Illegal selector - KIND value must be non-negative  (varpro.f90: 428)
F90-S-0081-Illegal selector - KIND value must be non-negative  (varpro.f90: 429)

Anyone else get further?

[thda0531]

Hi,

might be that an additional pre-compiler option (CPP_OPTIONS) in makefile.include helps:

-DnoQuadPrecision

This will switch 'qd' in base.F from
INTEGER, PARAMETER :: qd=SELECTED_REAL_KIND(30)
to
INTEGER, PARAMETER :: qd=SELECTED_REAL_KIND(10)

But, I don't know the impact of this change to the reliability of the results.

Btw., could you share your makefile.include with us??

Good luck.

[merzuk.kaltak]

Hello,
varpro.F is used inside the minimax module and requires quad-precision to determine quadratures for RPA and low-scaling RPA/GW calculations.
It seems your compiler is not supporting quad-precision. As a solution you may want to emulate quad-precision using the qd library of David H. Bailey.
You'll have to change the makefile.include file and add following lines (adapted to your installation of qd)

Code: Select all

#switch on qd-emulation using qd-lib
CPP_OPTIONS += -Dqd_emulate
# Software emulation of quadruple precsion
LLIBS      += -L/path/to/qd/install/lib -lqdmod -lqd
INCS       += -I/path/to/qd/install/include/qd

I suggest to take a look at ./arch/makefile.include.linux_pgi, which shows how to set these options for the PGI compiler suite (that also doesn't support quad-precision).

[lisg_sari]

Hello:

Hope everyone is doing well.

We are also interested in running VASP faster on the latest AMD EPYC processors. I recently tested both 32- and 64-core AMD Rome CPUs, and I first used our previously built binaries using the Intel compiler toolchains, which shows promising performance competitive against our 20-core Intel CPUs. However, I hope it can do better with AMD's own compilers and libraries.

I then built VASP using the recommended compiler flags for AOCC (verion 2.2, clang, clang++, flang), and for the AOCL libraries I tried both AMD provided (version 2.1, as version 2.2 was built on a very recent Ubuntu system, and we have CentOS 7.6) as well as my own built versions (version 2.2 built using AOCC 2.2). I was able to finish the build, but I had to adapt the preprocessor directive from the GNU or PGI compiler settings, which I am not quite familiar with and AOCC compilers do not seem to support these flags. The end result is that the binary always caused segmentation fault right after entering the SCF cycle, and I didn't further debug the binary. By the way, I used the latest OpenMPI version (4.0.5) for the build, in case that might be the reason, and the QD libraries had to be built and linked.

As the Intel compilers are well known for their high efficiency, I doubt the AOCC/AOCL combination will do better even on these AMD systems, but just hope someone can build usable executables with better performance using the AMD compiler systems.

Cheers,

Shenggang Li.