My Community

Posted: **Mon May 10, 2021 5:35 pm**

Which would be the appropriate starting makefile to use to compile 6.2 on nodes comprised of AMD EPYC 7742 64-Core Processors, running 5.3.0-62-generic #56~18.04.1-Ubuntu SMP Wed Jun 24 16:17:03 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux? The CPUs are AMD EPYC 7742 64-Core Processors. I want to compile a CPU only version and an OpenACC version.

Posted: **Wed May 12, 2021 4:23 pm**

Hi!

I am working on a Linux Mint 20.1 machine with AMD Epyc 7402P processor (24 cores), so it looks like a very similar setup:

Code: Select all

Linux 5.4.0-70-generic #78-Ubuntu SMP Fri Mar 19 13:29:52 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

I have not yet tried anything special but starting from the makefile.include

Code: Select all

arch/makefile.include.linux_gnu_omp

I could compile the CPU version with gfortran 9.3 and the system libraries. You may need to install some packages:

Code: Select all

sudo apt install libscalapack-openmpi-dev libfftw3-dev libopenblas-dev

With this setup the "fast" testsuite runs without errors. Here is the makefile.include I used:

Code: Select all

# Precompiler options
CPP_OPTIONS= -DHOST=\"LinuxGNU\" \
             -DMPI -DMPI_BLOCK=8000 -Duse_collective \
             -DscaLAPACK \
             -DCACHE_SIZE=4000 \
             -Davoidalloc \
             -Dvasp6 \
             -Duse_bse_te \
             -Dtbdyn \
             -Dfock_dblbuf \
             -D_OPENMP

CPP        = gcc -E -P -C -w $*$(FUFFIX) >$*$(SUFFIX) $(CPP_OPTIONS)

FC         = mpif90 -fopenmp
FCL        = mpif90 -fopenmp

FREE       = -ffree-form -ffree-line-length-none

FFLAGS     = -w -march=native
OFLAG      = -O2
OFLAG_IN   = $(OFLAG)
DEBUG      = -O0

BLAS       = -lopenblas
LAPACK     =
BLACS      = 
SCALAPACK  = -lscalapack-openmpi $(BLACS)

LLIBS      = $(SCALAPACK) $(LAPACK) $(BLAS)

FFTW       ?= 
LLIBS      += -lfftw3 -lfftw3_omp
INCS       = -I/usr/include

OBJECTS    = fftmpiw.o fftmpi_map.o  fftw3d.o  fft3dlib.o

OBJECTS_O1 += fftw3d.o fftmpi.o fftmpiw.o
OBJECTS_O2 += fft3dlib.o

# For what used to be vasp.5.lib
CPP_LIB    = $(CPP)
FC_LIB     = $(FC)
CC_LIB     = gcc
CFLAGS_LIB = -O
FFLAGS_LIB = -O1
FREE_LIB   = $(FREE)

OBJECTS_LIB= linpack_double.o getshmem.o

# For the parser library
CXX_PARS   = g++
LLIBS      += -lstdc++

### For the fft library
##CXX_FFTLIB = g++ -fopenmp -std=c++11 -DFFTLIB_THREADSAFE 
##INCS_FFTLIB= -I./include -I$(FFTW)/include
##LIBS       += fftlib
##LLIBS      += -ldl

# Normally no need to change this
SRCDIR     = ../../src
BINDIR     = ../../bin

I am currently looking into the AMD Optimizing Compiler (AOCC, https://developer.amd.com/amd-aocc/) and Libraries (AOCL, https://developer.amd.com/amd-aocl/) but I cannot give you a recommendation yet.

Please also have a look at the Wiki page regarding the installation of VASP (in particular for the OpenACC version): wiki/index.php/Installing_VASP.6.X.X

Posted: **Thu Sep 09, 2021 5:31 am**

Thank you @andreas.singraber for the detailed reply. I was wondering if you / the VASP team have/s made progress on a makefile for AMD Optimizing Compiler 3.1 and AMD Optimizing Libraries 3.0? Like @david_keller who started this thread, I also need to optimize VASP for the AMD EPYC family of processors (including 7742). Thank you so much!

Posted: **Thu Sep 30, 2021 10:14 am**

Hi!

Unfortunately I didn't find the time to do this yet.. however, it is on my table and getting a more pressing issue. I cannot promise a date for a satisfying answer but I will be working on the performance tests in the coming days and weeks. I'm sorry I have no answer to your questions at this point..

Best,
Andreas Singraber

Posted: **Tue Oct 05, 2021 4:34 am**

Thanks a lot for committing to this as well Andreas! There seems to be a growing group that needs a solid solution for VASP on AMD EPYC. I am trying out several possible solutions myself, but I have not been successful to date. Will post my solution here when I find it!

Posted: **Mon Dec 13, 2021 5:17 pm**

Hello again!

First of all, I'm sorry for the long delay... it turned out to be quite tricky to get good performance on these machines but I can report that in the meantime we already found a working setup! Unfortunately, we had to alter some of our internal OpenMP usage and therefore a simple solution with only a new makefile.include will not be sufficient. However, the source code changes and a recommended makefile.include for the AMD EPYC architecture will be delivered with the upcoming VASP release (in the very near future)... please stay tuned!!

Best,

Andreas Singraber

Posted: **Mon Dec 13, 2021 9:45 pm**

Thanks Andreas. We are looking forward to this new release with support for AMD EPYC 7742.

Posted: **Mon Dec 20, 2021 4:39 am**

Hi,

I also managed to compile the vasp6.2.1 with AOCC clang and tried to link them to the AOCL libraries with every optimisation flag I know. Unfortunately it is just slower than using the Intel compilers... Maybe just looking forward to the next version

My Community

Installing VASP CPU only version on AMD EPYC 7742

Installing VASP CPU only version on AMD EPYC 7742

Re: Installing VASP CPU only version on AMD EPYC 7742

Re: Installing VASP CPU only version on AMD EPYC 7742

Re: Installing VASP CPU only version on AMD EPYC 7742

Re: Installing VASP CPU only version on AMD EPYC 7742

Re: Installing VASP CPU only version on AMD EPYC 7742

Re: Installing VASP CPU only version on AMD EPYC 7742

Re: Installing VASP CPU only version on AMD EPYC 7742