Is there anyone who can use pinned memory option with xl compiler?

[guiyang_huang1]

IBM power9 with 6 V100 gpu on summit of olcf.

Using pgi compiler, pinned memory option can be used without error.

Using gnu or xl compiler, pinned memory option can not be used without error, even if -O0 is used for all compilation options, including the nvcc compiler options.

Using gnu compiler, if only one cpu and one gpu are used, the job can run without error.

Using xl compiler, even if only one cpu and one gpu are used, the job can not run without error.

The speed of the gpu vasp using xl compiler is the fastest, even if it can not use the pinned memory options.

The gpu vasp using pgi compiler is the second fast, mainly because it can use the pinned memory options. The pinned memory option has noticeable positive effects if the system is not too small.

The gnu vasp using gnu compiler is the slowest, mainly because it can not use the pinned memory options.

In addition, magma has a very large effects on the performance of gpu vasp.
But no matter whether magma is used, for gnu and xl compiler, the pinned memory options can not be used. It would show errors when vasp is running.
The error seems to be related to the free() function. It tried to free the same place for many times.

For xl compiler with magma, trailing underscore needs to be added to the magma functions. Otherwise, it will use the wrong function, and show no error for the compilation.

[prodenas]

What kind of error you are getting in the compilation with pinned memory using XLF?