VASP GPU compilation error

[tvruwink]

Hi -
Trying to build Vasp 5.4.4 GPU version, and I'm running into compilation errors.

I was able to build the standard Vasp binaries no problem. This is on a CentOS7.5 box, with a NVIDIA Volta GPU. Drivers are v396.37

The error I am seeing when I run 'make gpu' is:

Code: Select all

...
Error: 'x' argument of 'c_sizeof' intrinsic at (1) must be an interoperable data entity
xclib_grad_gpu.f90:4079.46:

      call cublas_get_vector(NP, int(c_sizeof(DWORK(1)),c_int),  GPU_DWORK, 1,
                                              1
Error: 'x' argument of 'c_sizeof' intrinsic at (1) must be an interoperable data entity
make[2]: *** [xclib_grad_gpu.o] Error 1

I've got CUDA9.2 installed.

Makefile is

Code: Select all

# Precompiler options
CPP_OPTIONS= -DHOST=\"LinuxGNU\" \
             -DMPI -DMPI_BLOCK=8000 \
             -Duse_collective \
             -DscaLAPACK \
             -DCACHE_SIZE=4000 \
             -Davoidalloc \
             -Duse_bse_te \
             -Dtbdyn \
             -Duse_shmem

CPP        = gcc -E -P -C -w $*$(FUFFIX) >$*$(SUFFIX) $(CPP_OPTIONS)

FC         = mpif90
FCL        = mpif90

FREE       = -ffree-form -ffree-line-length-none

FFLAGS     = -w
OFLAG      = -O2
OFLAG_IN   = $(OFLAG)
DEBUG      = -O0

LIBDIR     = /software/OpenBLAS-2.19-cent7/lib
BLAS       = -L$(LIBDIR) -lopenblas
LAPACK     = -L$(LIBDIR) -llapack
BLACS      =
SCALAPACK  = -L/usr/local/lib -lscalapack $(BLACS)

LLIBS      = $(SCALAPACK) $(LAPACK) $(BLAS)

# Wannier90
LLIBS      += /usr/local/lib/libwannier.a

FFTW       ?= /usr/local
LLIBS      += -L$(FFTW)/lib -lfftw3
INCS       = -I$(FFTW)/include

OBJECTS    = fftmpiw.o fftmpi_map.o  fftw3d.o  fft3dlib.o

OBJECTS_O1 += fftw3d.o fftmpi.o fftmpiw.o
OBJECTS_O2 += fft3dlib.o

# For what used to be vasp.5.lib
CPP_LIB    = $(CPP)
FC_LIB     = $(FC)
CC_LIB     = gcc
CFLAGS_LIB = -O
FFLAGS_LIB = -O1
FREE_LIB   = $(FREE)

OBJECTS_LIB= linpack_double.o getshmem.o

# For the parser library
CXX_PARS   = g++

LIBS       += parser
LLIBS      += -Lparser -lparser -lstdc++

# Normally no need to change this
SRCDIR     = ../../src
BINDIR     = ../../bin


#================================================
# GPU Stuff

CPP_GPU    = -DCUDA_GPU -DRPROMU_CPROJ_OVERLAP -DUSE_PINNED_MEMORY -DCUFFT_MIN=28 -UscaLAPACK

OBJECTS_GPU = fftmpiw.o fftmpi_map.o fft3dlib.o fftw3d_gpu.o fftmpiw_gpu.o

CC         = gcc
CXX        = g++

CFLAGS     = -fPIC -DADD_ -Wall -openmp -DMAGMA_WITH_MKL -DMAGMA_SETAFFINITY -DGPUSHMEM=300 -DHAVE_CUBLAS

CUDA_ROOT  ?= /usr/local/cuda
NVCC       := $(CUDA_ROOT)/bin/nvcc -ccbin=gcc
CUDA_LIB   := -L$(CUDA_ROOT)/lib64 -lnvToolsExt -lcudart -lcuda -lcufft -lcublas

GENCODE_ARCH    := -gencode=arch=compute_30,code=\"sm_30,compute_30\" \
                   -gencode=arch=compute_35,code=\"sm_35,compute_35\" \
                   -gencode=arch=compute_60,code=\"sm_60,compute_60\"

MPI_INC    = /software/openmpi-3.1.2/include

[guiyang_huang1]

I don't have this problem for the vasp gpu compilation.

But the source I have needs to be revised.
I need to replace the c_size_t by c_int, and replace COVL by c_loc(COVL). I can not remember the exact name. But the compiler would tell you the type of parameter is mismatch. The compiler would tell you the place.
I found out the solution by google. It mentioned that the c_int, and c_loc. I check the source code, and found in the other place, c_int and c_loc() are used for the function.

c_size_t ------------->c_int
COVL ------------->c_loc(COVL)

I have checked the source code at the place you mentioned, and found it is the same.
It is not the place I have revised the code.

I think you can try to use pgi compiler to compile the gpu vasp.
gnu compiler seems to be unable to use the pinned memory. I don't know whether it would be also true for your system.

If you want to use gnu compiler, you can delete the pinned memory options first. When all things are done, you can add it to check whether it can be used for your system. In the system I used (IBM power9), pinned memory option can not be used for gnu and xl compiler, but can be used for pgi compiler.