VASP 6.3.0 fftw multiple reference during linking
Posted: Wed Feb 23, 2022 1:06 pm
Hi,
I'm using ubuntu20.04 with nvhpc22.2. I am facing some strange issues with multiple reference and undefined reference during linking stage
error message:
/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-f77api.o): in function `dfftw_init_threads_':
/home/dx/repo/fftw-3.3.10/threads/f77funcs.h:37: multiple definition of `dfftw_init_threads_'; fftlib.o:/home/dx/repo/vasp-6.3.0-nvhpc222/build/std/fftlib/src/fftlib.cpp:27: first defined here
/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-f77api.o): in function `dfftw_cleanup_threads_':
/home/dx/repo/fftw-3.3.10/threads/f77funcs.h:42: multiple definition of `dfftw_cleanup_threads_'; fftlib.o:/home/dx/repo/vasp-6.3.0-nvhpc222/build/std/fftlib/src/fftlib.cpp:82: first defined here
/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-api.o): in function `fftw_init_threads':
/home/dx/repo/fftw-3.3.10/threads/api.c:40: multiple definition of `fftw_init_threads'; fftlib.o:/home/dx/repo/vasp-6.3.0-nvhpc222/build/std/fftlib/src/fftlib.cpp:12: first defined here
/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-api.o): in function `fftw_cleanup_threads':
/home/dx/repo/fftw-3.3.10/threads/api.c:61: multiple definition of `fftw_cleanup_threads'; fftlib.o:/home/dx/repo/vasp-6.3.0-nvhpc222/build/std/fftlib/src/fftlib.cpp:67: first defined here/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-api.o): in function `fftw_plan_with_nthreads':
/home/dx/repo/fftw-3.3.10/threads/api.c:71: multiple definition of `fftw_plan_with_nthreads'; fftlib.o:/home/dx/repo/vasp-6.3.0-nvhpc222/build/std/fftlib/src/fftlib.cpp:32: first defined here
/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-api.o): in function `fftw_init_threads':
/home/dx/repo/fftw-3.3.10/threads/api.c:51: undefined reference to `fftw_the_planner'
/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-api.o): in function `fftw_plan_with_nthreads':
/home/dx/repo/fftw-3.3.10/threads/api.c:79: undefined reference to `fftw_the_planner'
/usr/bin/ld: /home/dx/repo/fftw-3.3.10/threads/api.c:80: undefined reference to `fftw_imax'
/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-api.o): in function `fftw_planner_nthreads':
/home/dx/repo/fftw-3.3.10/threads/api.c:85: undefined reference to `fftw_the_planner'
/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-api.o): in function `threads_register_hooks':
/home/dx/repo/fftw-3.3.10/threads/api.c:28: undefined reference to `fftw_mksolver_ct_hook'
/usr/bin/ld: /home/dx/repo/fftw-3.3.10/threads/api.c:29: undefined reference to `fftw_mksolver_hc2hc_hook'
/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-api.o): in function `threads_unregister_hooks':
/home/dx/repo/fftw-3.3.10/threads/api.c:34: undefined reference to `fftw_mksolver_ct_hook'
/usr/bin/ld: /home/dx/repo/fftw-3.3.10/threads/api.c:35: undefined reference to `fftw_mksolver_hc2hc_hook'
/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-conf.o): in function `fftw_threads_conf_standard':
/home/dx/repo/fftw-3.3.10/threads/conf.c:35: undefined reference to `fftw_solvtab_exec'
/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-dft-vrank-geq1.o): in function `awake':
/home/dx/repo/fftw-3.3.10/threads/dft-vrank-geq1.c:77: undefined reference to `fftw_plan_awake'
/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-dft-vrank-geq1.o): in function `destroy':
/home/dx/repo/fftw-3.3.10/threads/dft-vrank-geq1.c:85: undefined reference to `fftw_plan_destroy_internal'
/usr/bin/ld: /home/dx/repo/fftw-3.3.10/threads/dft-vrank-geq1.c:86: undefined reference to `fftw_ifree'
/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-dft-vrank-geq1.o): in function `mkplan':
/home/dx/repo/fftw-3.3.10/threads/dft-vrank-geq1.c:164: undefined reference to `fftw_malloc_plain'
/usr/bin/ld: /home/dx/repo/fftw-3.3.10/threads/dft-vrank-geq1.c:167: undefined reference to `fftw_tensor_copy'
/usr/bin/ld: /home/dx/repo/fftw-3.3.10/threads/dft-vrank-geq1.c:171: undefined reference to `fftw_mkproblem_dft'
/usr/bin/ld: /home/dx/repo/fftw-3.3.10/threads/dft-vrank-geq1.c:174: undefined reference to `fftw_mkplan_d'
/usr/bin/ld: /home/dx/repo/fftw-3.3.10/threads/dft-vrank-geq1.c:177: undefined reference to `fftw_tensor_destroy'
/usr/bin/ld: /home/dx/repo/fftw-3.3.10/threads/dft-vrank-geq1.c:179: undefined reference to `fftw_mkplan_dft'
/usr/bin/ld: /home/dx/repo/fftw-3.3.10/threads/dft-vrank-geq1.c:187: undefined reference to `fftw_ops_zero'
/usr/bin/ld: /home/dx/repo/fftw-3.3.10/threads/dft-vrank-geq1.c:190: undefined reference to `fftw_ops_add2'
/usr/bin/ld: /home/dx/repo/fftw-3.3.10/threads/dft-vrank-geq1.c:199: undefined reference to `fftw_plan_destroy_internal'
/usr/bin/ld: /home/dx/repo/fftw-3.3.10/threads/dft-vrank-geq1.c:200: undefined reference to `fftw_ifree'
/usr/bin/ld: /home/dx/repo/fftw-3.3.10/threads/dft-vrank-geq1.c:202: undefined reference to `fftw_tensor_destroy'
/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-dft-vrank-geq1.o): in function `fftw_dft_thr_vrank_geq1_register':
/home/dx/repo/fftw-3.3.10/threads/dft-vrank-geq1.c:223: undefined reference to `fftw_solver_register'
/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-dft-vrank-geq1.o): in function `pickdim':
/home/dx/repo/fftw-3.3.10/threads/dft-vrank-geq1.c:104: undefined reference to `fftw_pickdim'
/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-dft-vrank-geq1.o): in function `mksolver':
/home/dx/repo/fftw-3.3.10/threads/dft-vrank-geq1.c:209: undefined reference to `fftw_mksolver'/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-dft-vrank-geq1.o):(.data+0x0): undefined reference to `fftw_dft_solve'
/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-ct.o): in function `awake':
/home/dx/repo/fftw-3.3.10/threads/ct.c:87: undefined reference to `fftw_plan_awake'
/usr/bin/ld: /home/dx/repo/fftw-3.3.10/threads/ct.c:89: undefined reference to `fftw_plan_awake'
/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-ct.o): in function `destroy':
/home/dx/repo/fftw-3.3.10/threads/ct.c:96: undefined reference to `fftw_plan_destroy_internal'/usr/bin/ld: /home/dx/repo/fftw-3.3.10/threads/ct.c:98: undefined reference to `fftw_plan_destroy_internal'
/usr/bin/ld: /home/dx/repo/fftw-3.3.10/threads/ct.c:99: undefined reference to `fftw_ifree'
/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-ct.o): in function `mkplan':
/home/dx/repo/fftw-3.3.10/threads/ct.c:131: undefined reference to `fftw_ct_applicable'
/usr/bin/ld: /home/dx/repo/fftw-3.3.10/threads/ct.c:137: undefined reference to `fftw_choose_radix'
/usr/bin/ld: /home/dx/repo/fftw-3.3.10/threads/ct.c:140: undefined reference to `fftw_tensor_tornk1'
...
makefile
# Default precompiler options
CPP_OPTIONS = -DHOST=\"LinuxNV\" \
-DMPI -DMPI_BLOCK=8000 -Duse_collective \
-DscaLAPACK \
-DCACHE_SIZE=4000 \
-Davoidalloc \
-Dvasp6 \
-Duse_bse_te \
-Dtbdyn \
-Dqd_emulate \
-Dfock_dblbuf \
-D_OPENMP \
-D_OPENACC \
-DUSENCCL -DUSENCCLP2P
CPP = nvfortran -Mpreprocess -Mfree -Mextend -E $(CPP_OPTIONS) $*$(FUFFIX) > $*$(SUFFIX)
# N.B.: you might need to change the cuda-version here
# to one that comes with your NVIDIA-HPC SDK
FC = mpif90 -acc -gpu=cc70,cuda11.0 -mp
FCL = mpif90 -acc -gpu=cc70,cuda11.0 -mp -c++libs
FREE = -Mfree
FFLAGS = -Mbackslash -Mlarge_arrays
OFLAG = -fast
DEBUG = -Mfree -O0 -traceback
OBJECTS = fftmpiw.o fftmpi_map.o fftw3d.o fft3dlib.o
LLIBS = -cudalib=cublas,cusolver,cufft,nccl -cuda
# Redefine the standard list of O1 and O2 objects
SOURCE_O1 := pade_fit.o
SOURCE_O2 := pead.o
# For what used to be vasp.5.lib
CPP_LIB = $(CPP)
FC_LIB = nvfortran
CC_LIB = nvc -w
CFLAGS_LIB = -O
FFLAGS_LIB = -O1 -Mfixed
FREE_LIB = $(FREE)
OBJECTS_LIB = linpack_double.o
# For the parser library
CXX_PARS = nvc++ --no_warnings
##
## Customize as of this point! Of course you may change the preceding
## part of this file as well if you like, but it should rarely be
## necessary ...
##
# Specify your NV HPC-SDK installation (mandatory)
#... first try to set it automatically
NVROOT =$(shell which nvfortran | awk -F /compilers/bin/nvfortran '{ print $$1 }')
# If the above fails, then NVROOT needs to be set manually
#NVHPC ?= /opt/nvidia/hpc_sdk
#NVVERSION = 21.11
#NVROOT = $(NVHPC)/Linux_x86_64/$(NVVERSION)
# Software emulation of quadruple precsion (mandatory)
QD ?= $(NVROOT)/compilers/extras/qd
LLIBS += -L$(QD)/lib -lqdmod -lqd
INCS += -I$(QD)/include/qd
# BLAS (mandatory)
BLAS = -lblas
# LAPACK (mandatory)
LAPACK = -llapack
# scaLAPACK (mandatory)
SCALAPACK = -Mscalapack
LLIBS += $(SCALAPACK) $(LAPACK) $(BLAS)
# FFTW (mandatory)
FFTW_ROOT = /home/dx/app/fftw3310
LLIBS += -L$(FFTW_ROOT)/lib -lfftw3 -lfftw3_omp
INCS += -I$(FFTW_ROOT)/include
# HDF5-support (optional but strongly recommended)
#CPP_OPTIONS+= -DVASP_HDF5
#HDF5_ROOT ?= /path/to/your/hdf5/installation
#LLIBS += -L$(HDF5_ROOT)/lib -lhdf5_fortran
#INCS += -I$(HDF5_ROOT)/include
# For the VASP-2-Wannier90 interface (optional)
#CPP_OPTIONS += -DVASP2WANNIER90
#WANNIER90_ROOT ?= /path/to/your/wannier90/installation
#LLIBS += -L$(WANNIER90_ROOT)/lib -lwannier
# For the fftlib library (experimental)
CPP_OPTIONS+= -Dsysv
FCL += fftlib.o
CXX_FFTLIB = nvc++ -mp --no_warnings -std=c++11 -DFFTLIB_THREADSAFE
INCS_FFTLIB = -I./include -I$(FFTW_ROOT)/include
LIBS += fftlib
LLIBS += -ldl
This issue disappear when I disable the experimental fftlib library or linking with intel mkl and fftw
My fftw is configured with nvhpc by:
CC=nvc CXX=nvc++ FC=nvfortran ./configure --enable-openmp --enable-mpi
I'm using ubuntu20.04 with nvhpc22.2. I am facing some strange issues with multiple reference and undefined reference during linking stage
error message:
/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-f77api.o): in function `dfftw_init_threads_':
/home/dx/repo/fftw-3.3.10/threads/f77funcs.h:37: multiple definition of `dfftw_init_threads_'; fftlib.o:/home/dx/repo/vasp-6.3.0-nvhpc222/build/std/fftlib/src/fftlib.cpp:27: first defined here
/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-f77api.o): in function `dfftw_cleanup_threads_':
/home/dx/repo/fftw-3.3.10/threads/f77funcs.h:42: multiple definition of `dfftw_cleanup_threads_'; fftlib.o:/home/dx/repo/vasp-6.3.0-nvhpc222/build/std/fftlib/src/fftlib.cpp:82: first defined here
/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-api.o): in function `fftw_init_threads':
/home/dx/repo/fftw-3.3.10/threads/api.c:40: multiple definition of `fftw_init_threads'; fftlib.o:/home/dx/repo/vasp-6.3.0-nvhpc222/build/std/fftlib/src/fftlib.cpp:12: first defined here
/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-api.o): in function `fftw_cleanup_threads':
/home/dx/repo/fftw-3.3.10/threads/api.c:61: multiple definition of `fftw_cleanup_threads'; fftlib.o:/home/dx/repo/vasp-6.3.0-nvhpc222/build/std/fftlib/src/fftlib.cpp:67: first defined here/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-api.o): in function `fftw_plan_with_nthreads':
/home/dx/repo/fftw-3.3.10/threads/api.c:71: multiple definition of `fftw_plan_with_nthreads'; fftlib.o:/home/dx/repo/vasp-6.3.0-nvhpc222/build/std/fftlib/src/fftlib.cpp:32: first defined here
/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-api.o): in function `fftw_init_threads':
/home/dx/repo/fftw-3.3.10/threads/api.c:51: undefined reference to `fftw_the_planner'
/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-api.o): in function `fftw_plan_with_nthreads':
/home/dx/repo/fftw-3.3.10/threads/api.c:79: undefined reference to `fftw_the_planner'
/usr/bin/ld: /home/dx/repo/fftw-3.3.10/threads/api.c:80: undefined reference to `fftw_imax'
/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-api.o): in function `fftw_planner_nthreads':
/home/dx/repo/fftw-3.3.10/threads/api.c:85: undefined reference to `fftw_the_planner'
/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-api.o): in function `threads_register_hooks':
/home/dx/repo/fftw-3.3.10/threads/api.c:28: undefined reference to `fftw_mksolver_ct_hook'
/usr/bin/ld: /home/dx/repo/fftw-3.3.10/threads/api.c:29: undefined reference to `fftw_mksolver_hc2hc_hook'
/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-api.o): in function `threads_unregister_hooks':
/home/dx/repo/fftw-3.3.10/threads/api.c:34: undefined reference to `fftw_mksolver_ct_hook'
/usr/bin/ld: /home/dx/repo/fftw-3.3.10/threads/api.c:35: undefined reference to `fftw_mksolver_hc2hc_hook'
/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-conf.o): in function `fftw_threads_conf_standard':
/home/dx/repo/fftw-3.3.10/threads/conf.c:35: undefined reference to `fftw_solvtab_exec'
/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-dft-vrank-geq1.o): in function `awake':
/home/dx/repo/fftw-3.3.10/threads/dft-vrank-geq1.c:77: undefined reference to `fftw_plan_awake'
/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-dft-vrank-geq1.o): in function `destroy':
/home/dx/repo/fftw-3.3.10/threads/dft-vrank-geq1.c:85: undefined reference to `fftw_plan_destroy_internal'
/usr/bin/ld: /home/dx/repo/fftw-3.3.10/threads/dft-vrank-geq1.c:86: undefined reference to `fftw_ifree'
/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-dft-vrank-geq1.o): in function `mkplan':
/home/dx/repo/fftw-3.3.10/threads/dft-vrank-geq1.c:164: undefined reference to `fftw_malloc_plain'
/usr/bin/ld: /home/dx/repo/fftw-3.3.10/threads/dft-vrank-geq1.c:167: undefined reference to `fftw_tensor_copy'
/usr/bin/ld: /home/dx/repo/fftw-3.3.10/threads/dft-vrank-geq1.c:171: undefined reference to `fftw_mkproblem_dft'
/usr/bin/ld: /home/dx/repo/fftw-3.3.10/threads/dft-vrank-geq1.c:174: undefined reference to `fftw_mkplan_d'
/usr/bin/ld: /home/dx/repo/fftw-3.3.10/threads/dft-vrank-geq1.c:177: undefined reference to `fftw_tensor_destroy'
/usr/bin/ld: /home/dx/repo/fftw-3.3.10/threads/dft-vrank-geq1.c:179: undefined reference to `fftw_mkplan_dft'
/usr/bin/ld: /home/dx/repo/fftw-3.3.10/threads/dft-vrank-geq1.c:187: undefined reference to `fftw_ops_zero'
/usr/bin/ld: /home/dx/repo/fftw-3.3.10/threads/dft-vrank-geq1.c:190: undefined reference to `fftw_ops_add2'
/usr/bin/ld: /home/dx/repo/fftw-3.3.10/threads/dft-vrank-geq1.c:199: undefined reference to `fftw_plan_destroy_internal'
/usr/bin/ld: /home/dx/repo/fftw-3.3.10/threads/dft-vrank-geq1.c:200: undefined reference to `fftw_ifree'
/usr/bin/ld: /home/dx/repo/fftw-3.3.10/threads/dft-vrank-geq1.c:202: undefined reference to `fftw_tensor_destroy'
/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-dft-vrank-geq1.o): in function `fftw_dft_thr_vrank_geq1_register':
/home/dx/repo/fftw-3.3.10/threads/dft-vrank-geq1.c:223: undefined reference to `fftw_solver_register'
/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-dft-vrank-geq1.o): in function `pickdim':
/home/dx/repo/fftw-3.3.10/threads/dft-vrank-geq1.c:104: undefined reference to `fftw_pickdim'
/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-dft-vrank-geq1.o): in function `mksolver':
/home/dx/repo/fftw-3.3.10/threads/dft-vrank-geq1.c:209: undefined reference to `fftw_mksolver'/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-dft-vrank-geq1.o):(.data+0x0): undefined reference to `fftw_dft_solve'
/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-ct.o): in function `awake':
/home/dx/repo/fftw-3.3.10/threads/ct.c:87: undefined reference to `fftw_plan_awake'
/usr/bin/ld: /home/dx/repo/fftw-3.3.10/threads/ct.c:89: undefined reference to `fftw_plan_awake'
/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-ct.o): in function `destroy':
/home/dx/repo/fftw-3.3.10/threads/ct.c:96: undefined reference to `fftw_plan_destroy_internal'/usr/bin/ld: /home/dx/repo/fftw-3.3.10/threads/ct.c:98: undefined reference to `fftw_plan_destroy_internal'
/usr/bin/ld: /home/dx/repo/fftw-3.3.10/threads/ct.c:99: undefined reference to `fftw_ifree'
/usr/bin/ld: /home/dx/app/fftw3310/lib/libfftw3_omp.a(libfftw3_omp_la-ct.o): in function `mkplan':
/home/dx/repo/fftw-3.3.10/threads/ct.c:131: undefined reference to `fftw_ct_applicable'
/usr/bin/ld: /home/dx/repo/fftw-3.3.10/threads/ct.c:137: undefined reference to `fftw_choose_radix'
/usr/bin/ld: /home/dx/repo/fftw-3.3.10/threads/ct.c:140: undefined reference to `fftw_tensor_tornk1'
...
makefile
# Default precompiler options
CPP_OPTIONS = -DHOST=\"LinuxNV\" \
-DMPI -DMPI_BLOCK=8000 -Duse_collective \
-DscaLAPACK \
-DCACHE_SIZE=4000 \
-Davoidalloc \
-Dvasp6 \
-Duse_bse_te \
-Dtbdyn \
-Dqd_emulate \
-Dfock_dblbuf \
-D_OPENMP \
-D_OPENACC \
-DUSENCCL -DUSENCCLP2P
CPP = nvfortran -Mpreprocess -Mfree -Mextend -E $(CPP_OPTIONS) $*$(FUFFIX) > $*$(SUFFIX)
# N.B.: you might need to change the cuda-version here
# to one that comes with your NVIDIA-HPC SDK
FC = mpif90 -acc -gpu=cc70,cuda11.0 -mp
FCL = mpif90 -acc -gpu=cc70,cuda11.0 -mp -c++libs
FREE = -Mfree
FFLAGS = -Mbackslash -Mlarge_arrays
OFLAG = -fast
DEBUG = -Mfree -O0 -traceback
OBJECTS = fftmpiw.o fftmpi_map.o fftw3d.o fft3dlib.o
LLIBS = -cudalib=cublas,cusolver,cufft,nccl -cuda
# Redefine the standard list of O1 and O2 objects
SOURCE_O1 := pade_fit.o
SOURCE_O2 := pead.o
# For what used to be vasp.5.lib
CPP_LIB = $(CPP)
FC_LIB = nvfortran
CC_LIB = nvc -w
CFLAGS_LIB = -O
FFLAGS_LIB = -O1 -Mfixed
FREE_LIB = $(FREE)
OBJECTS_LIB = linpack_double.o
# For the parser library
CXX_PARS = nvc++ --no_warnings
##
## Customize as of this point! Of course you may change the preceding
## part of this file as well if you like, but it should rarely be
## necessary ...
##
# Specify your NV HPC-SDK installation (mandatory)
#... first try to set it automatically
NVROOT =$(shell which nvfortran | awk -F /compilers/bin/nvfortran '{ print $$1 }')
# If the above fails, then NVROOT needs to be set manually
#NVHPC ?= /opt/nvidia/hpc_sdk
#NVVERSION = 21.11
#NVROOT = $(NVHPC)/Linux_x86_64/$(NVVERSION)
# Software emulation of quadruple precsion (mandatory)
QD ?= $(NVROOT)/compilers/extras/qd
LLIBS += -L$(QD)/lib -lqdmod -lqd
INCS += -I$(QD)/include/qd
# BLAS (mandatory)
BLAS = -lblas
# LAPACK (mandatory)
LAPACK = -llapack
# scaLAPACK (mandatory)
SCALAPACK = -Mscalapack
LLIBS += $(SCALAPACK) $(LAPACK) $(BLAS)
# FFTW (mandatory)
FFTW_ROOT = /home/dx/app/fftw3310
LLIBS += -L$(FFTW_ROOT)/lib -lfftw3 -lfftw3_omp
INCS += -I$(FFTW_ROOT)/include
# HDF5-support (optional but strongly recommended)
#CPP_OPTIONS+= -DVASP_HDF5
#HDF5_ROOT ?= /path/to/your/hdf5/installation
#LLIBS += -L$(HDF5_ROOT)/lib -lhdf5_fortran
#INCS += -I$(HDF5_ROOT)/include
# For the VASP-2-Wannier90 interface (optional)
#CPP_OPTIONS += -DVASP2WANNIER90
#WANNIER90_ROOT ?= /path/to/your/wannier90/installation
#LLIBS += -L$(WANNIER90_ROOT)/lib -lwannier
# For the fftlib library (experimental)
CPP_OPTIONS+= -Dsysv
FCL += fftlib.o
CXX_FFTLIB = nvc++ -mp --no_warnings -std=c++11 -DFFTLIB_THREADSAFE
INCS_FFTLIB = -I./include -I$(FFTW_ROOT)/include
LIBS += fftlib
LLIBS += -ldl
This issue disappear when I disable the experimental fftlib library or linking with intel mkl and fftw
My fftw is configured with nvhpc by:
CC=nvc CXX=nvc++ FC=nvfortran ./configure --enable-openmp --enable-mpi