Compiling CFDEM with g++ instead of mpic++

Hello all

I have a Suse Enterprise cluster system where LIGGGHTS-3.0.6 and OpenFOAM-3.0.x are successfully installed. Third-party gcc-4.8.1 was used to compile LIGGGHTS by Makefile.g++ file and OpenFOAM.

The details fo gcc, mpirun are as follows:
service0 cfdemParticleComp/Make> mpic++ mpic++: Command not found. service0 cfdemParticleComp/Make> which g++ /lustre/scratch/rsoni/OpenFOAM/ThirdParty-3.0.x/platforms/linux64/gcc-4.8.1/bin/g++ service0 cfdemParticleComp/Make> g++ --version g++ (OpenFOAM) 4.8.1 Copyright (C) 2013 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. service0 cfdemParticleComp/Make> mpirun --version MPI: invalid option: --version service0 cfdemParticleComp/Make> which mpirun /opt/sgi/mpt/mpt-2.01/bin/mpirun service0 cfdemParticleComp/Make>

The rsulst of cfdemSysTest is
service0 lagrangian/cfdemParticle> cfdemSysTest

*********************************************
* C F D E M (R) c o u p l i n g *
* *
* by DCS Computing GmbH *
* www.dcs-computing.com *
*********************************************

*********************************
CFDEM(R)coupling system settings:
*********************************
CFDEM_VERSION=PUBLIC
couple to OF_VERSION=3.0.x
compile option=Opt

check if paths are set correctly
valid:yes critical:yes - $CFDEM_PROJECT_DIR = /lustre/scratch/rsoni/CFDEM/CFDEMcoupling-PUBLIC-3.0.x
valid:yes critical:no - $CFDEM_PROJECT_USER_DIR = /lustre/scratch/rsoni/CFDEM/rsoni-PUBLIC-3.0.x
valid:yes critical:yes - $CFDEM_SRC_DIR = /lustre/scratch/rsoni/CFDEM/CFDEMcoupling-PUBLIC-3.0.x/src
valid:yes critical:yes - $CFDEM_SOLVER_DIR = /lustre/scratch/rsoni/CFDEM/CFDEMcoupling-PUBLIC-3.0.x/applications/solvers
valid:yes critical:yes - $CFDEM_TUT_DIR = /lustre/scratch/rsoni/CFDEM/CFDEMcoupling-PUBLIC-3.0.x/tutorials
valid:yes critical:yes - $CFDEM_LIGGGHTS_SRC_DIR = /lustre/scratch/rsoni/LIGGGHTS/LIGGGHTS-PUBLIC-360-g++/src
valid:yes critical:yes - $CFDEM_LPP_DIR = /lustre/scratch/rsoni/LIGGGHTS/LPP-Installation/src
valid:yes critical:yes - $CFDEM_ADD_LIBS_DIR = /lustre/scratch/rsoni/CFDEM/CFDEMcoupling-PUBLIC-3.0.x/src/lagrangian/cfdemParticle/etc/addLibs_universal
valid:yes critical:no - $CFDEM_PIZZA_DIR = /lustre/scratch/rsoni/CFDEM/pizza-9Oct15/src
valid:yes critical:no - $CFDEM_TEST_HARNESS_PATH = /lustre/scratch/rsoni/CFDEM/rsoni-PUBLIC-3.0.x/log/logFilesCFDEM-PUBLIC-3.0.x
valid:NO critical:no - $C3PO_SRC_DIR = does not exist

library names
$CFDEM_LIGGGHTS_LIB_NAME = lmp_g++
$CFDEM_LIB_NAME = lagrangianCFDEM-PUBLIC-3.0.x
$LD_LIBRARY_PATH = /lustre/scratch/rsoni/OpenFOAM/ThirdParty-3.0.x/platforms/linux64Gcc48/boost_1_54_0/lib:/lustre/scratch/rsoni/OpenFOAM/ThirdParty-3.0.x/platforms/linux64Gcc48/CGAL-4.7/lib:/lustre/scratch/rsoni/OpenFOAM/ThirdParty-3.0.x/platforms/linux64Gcc48/ParaView-4.4.0/lib/paraview-4.4:/lustre/scratch/rsoni/OpenFOAM/OpenFOAM-3.0.x/platforms/linux64Gcc48DPInt32Opt/lib/mpt-2.01:/lustre/scratch/rsoni/OpenFOAM/ThirdParty-3.0.x/platforms/linux64Gcc48DPInt32Opt/lib/mpt-2.01:/opt/sgi/mpt/mpt-2.01/lib:/lustre/scratch/rsoni/OpenFOAM/ThirdParty-3.0.x/platforms/linux64/mpc-1.0.1/lib:/lustre/scratch/rsoni/OpenFOAM/ThirdParty-3.0.x/platforms/linux64/mpfr-3.1.2/lib:/lustre/scratch/rsoni/OpenFOAM/ThirdParty-3.0.x/platforms/linux64/gmp-5.1.2/lib:/lustre/scratch/rsoni/OpenFOAM/ThirdParty-3.0.x/platforms/linux64/gcc-4.8.1/lib64:/lustre/scratch/rsoni/OpenFOAM/rsoni-3.0.x/platforms/linux64Gcc48DPInt32Opt/lib:/lustre/scratch/rsoni/OpenFOAM/site/3.0.x/platforms/linux64Gcc48DPInt32Opt/lib:/lustre/scratch/rsoni/OpenFOAM/OpenFOAM-3.0.x/platforms/linux64Gcc48DPInt32Opt/lib:/lustre/scratch/rsoni/OpenFOAM/ThirdParty-3.0.x/platforms/linux64Gcc48DPInt32Opt/lib:/lustre/scratch/rsoni/OpenFOAM/OpenFOAM-3.0.x/platforms/linux64Gcc48DPInt32Opt/lib/dummy:/opt/sgi/mpt/mpt-2.05/lib
$WM_NCOMPPROCS = 4
$WM_LABEL_SIZE = 32

Additional lib settings
VTK_PATH =
VTK_LIB =
VTK_INC =
CFDEM_ADD_LIB_PATHS = -L/chemkinReader/src -L/lib -L/lib64 -L -L/lib -L/lib
CFDEM_ADD_LIBS = -lboost_regex -lQt5Core -lhdf5_cpp -lhdf5 -lchemkinreader

*******************
g++:
/lustre/scratch/rsoni/OpenFOAM/ThirdParty-3.0.x/platforms/linux64/gcc-4.8.1/bin/g++
g++ (OpenFOAM) 4.8.1
Copyright (C) 2013 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

gcc:
/lustre/scratch/rsoni/OpenFOAM/ThirdParty-3.0.x/platforms/linux64/gcc-4.8.1/bin/gcc
gcc (OpenFOAM) 4.8.1
Copyright (C) 2013 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

mpic++:
which: no mpic++ in (/lustre/scratch/rsoni/OpenFOAM/ThirdParty-3.0.x/platforms/linux64Gcc48/ParaView-4.4.0/bin:/lustre/scratch/rsoni/OpenFOAM/ThirdParty-3.0.x/platforms/linux64Gcc48/cmake-3.2.1/bin:/opt/sgi/mpt/mpt-2.01/bin:/lustre/scratch/rsoni/OpenFOAM/ThirdParty-3.0.x/platforms/linux64/gcc-4.8.1/bin:/lustre/scratch/rsoni/OpenFOAM/rsoni-3.0.x/platforms/linux64Gcc48DPInt32Opt/bin:/lustre/scratch/rsoni/OpenFOAM/site/3.0.x/platforms/linux64Gcc48DPInt32Opt/bin:/lustre/scratch/rsoni/OpenFOAM/OpenFOAM-3.0.x/platforms/linux64Gcc48DPInt32Opt/bin:/lustre/scratch/rsoni/OpenFOAM/OpenFOAM-3.0.x/bin:/lustre/scratch/rsoni/OpenFOAM/OpenFOAM-3.0.x/wmake:/opt/sgi/mpt/mpt-2.05/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/bin/X11:/usr/X11R6/bin:/usr/games:/opt/c3/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin:/usr/lib/qt3/bin:/opt/sgi/sbin:/opt/sgi/bin:/opt/pbs/default/bin:/home1/rsoni/bin:/opt/pbs/default/bin:/home1/rsoni/bin:/opt/pbs/default/bin:/home1/rsoni/bin)
/lustre/scratch/rsoni/CFDEM/CFDEMcoupling-PUBLIC-3.0.x/src/lagrangian/cfdemParticle/etc/cfdemSystemTest.sh: line 86: mpic++: command not found
mpirun:
/opt/sgi/mpt/mpt-2.01/bin/mpirun
MPI: invalid option: --version
**********************
additional packages...
service0 lagrangian/cfdemParticle>

Now, as we can see I don't have mpic++ at the cluster, I think I have the option left to compile the program with gcc/g++ only. How can I instruct the CFDEM to do so.

When I am trying to compile solvers with cfdemCompCFDEMSol followed after cfdemCompLIG and cfdemCompCFDEMsrc, then I believe that the following part of screen output is causing the error:
-lm -o /lustre/scratch/rsoni/OpenFOAM/rsoni-3.0.x/platforms/linux64Gcc48DPInt32Opt/bin/cfdemSolverPiso /lustre/scratch/rsoni/OpenFOAM/ThirdParty-3.0.x/platforms/linux64/gcc-4.8.1/lib/gcc/x86_64-unknown-linux-gnu/4.8.1/../../../../x86_64-unknown-linux-gnu/bin/ld: cannot find -llagrangianCFDEM-PUBLIC-3.0.x /lustre/scratch/rsoni/OpenFOAM/ThirdParty-3.0.x/platforms/linux64/gcc-4.8.1/lib/gcc/x86_64-unknown-linux-gnu/4.8.1/../../../../x86_64-unknown-linux-gnu/bin/ld: cannot find -lboost_regex /lustre/scratch/rsoni/OpenFOAM/ThirdParty-3.0.x/platforms/linux64/gcc-4.8.1/lib/gcc/x86_64-unknown-linux-gnu/4.8.1/../../../../x86_64-unknown-linux-gnu/bin/ld: cannot find -lQt5Core /lustre/scratch/rsoni/OpenFOAM/ThirdParty-3.0.x/platforms/linux64/gcc-4.8.1/lib/gcc/x86_64-unknown-linux-gnu/4.8.1/../../../../x86_64-unknown-linux-gnu/bin/ld: cannot find -lhdf5_cpp /lustre/scratch/rsoni/OpenFOAM/ThirdParty-3.0.x/platforms/linux64/gcc-4.8.1/lib/gcc/x86_64-unknown-linux-gnu/4.8.1/../../../../x86_64-unknown-linux-gnu/bin/ld: cannot find -lhdf5 /lustre/scratch/rsoni/OpenFOAM/ThirdParty-3.0.x/platforms/linux64/gcc-4.8.1/lib/gcc/x86_64-unknown-linux-gnu/4.8.1/../../../../x86_64-unknown-linux-gnu/bin/ld: cannot find -lchemkinreader collect2: error: ld returned 1 exit status make: *** [/lustre/scratch/rsoni/OpenFOAM/rsoni-3.0.x/platforms/linux64Gcc48DPInt32Opt/bin/cfdemSolverPiso] Error 1 compiling cfdemSolverIB

// log_compileCFDEMcoupling_cfdemSolverIB_cfdemSolverIB-2017-01-25-12:18 //

I tried checking but hierarchy of folders /lustre/scratch/rsoni/OpenFOAM/ThirdParty-3.0.x/platforms/linux64/gcc-4.8.1/lib/gcc/x86_64-unknown-linux-gnu/4.8.1/../../../../x86_64-unknown-linux-gnu/bin/ld does not exist.

My errors also include:
could not open file mpi.h for source file phaseChangeModel/phaseChangeModel.C due to No such file or directory

Can any one help me troubleshooting this. I am not sure whether it is because of the absence of mpic++

Attachment	Size
makefile.g.txt	2.71 KB

Forums:

CFDEM®coupling - User Forum

j-kerbl | Thu, 01/26/2017 - 13:15

Hi Rahul,

Hi Rahul,

I think you have two options here:
Either install OpenMPI via the ThirdParty, as you installed gcc
or get it to work with MPT. Since you need the fPic flags in LIGGGHTS for the combination with CFDEMcoupling, you'll need to modify the g++ Makefile. (compare it with e.g. fedora_fpic). The first step is to get LIGGGHTS to work with fPic and compile it with cfdemCompLIG. Second step is to compile the libraries of the coupling with cfdemCompCFDEMsrc.

I think the easier way is to get OpenMPI to run on the cluster. (I suggest you get in touch with the admin team)

Cheers,
Josef

rahulsoni | Thu, 01/26/2017 - 15:45

LIGGGHTS with g++ and fPIC

Hello Josef and thanks for help

Josef I think my LIGGGHTS is already compiled with fPIC flag. A copy of Makefile.g++ is attached for reference. I have the steps of cfdemCompLIG and cfdemCompCFDEMsrc, and then I got the shared results. I do agree that I have either of the two options OpenMPI or MPT, but I think going with OpenMPI will again force me to recompile LIGGGHTS and OpenFOAM. And, OpenFOAM whose installation was a nightmare, I am afraid how much more time it will again take. So, if possible let me know if it is ok going with MPT.

j-kerbl | Wed, 02/01/2017 - 17:44

Hi Rahul,

Hi Rahul,

I don't know the standard MPT implementations very well.
Usually you need to modify OpenFOAMs compilation a bit, to compile it with MPT. So i suggest to do some digging in the OpenFOAM installation.
Specifically the entry WM_MPLIB of OFs bashrc and the corresponding entry in case "$WM_MPLIB" in OpenFOAM-x.x/etc/config/settings.sh are interesting.

What you definitely need to check, is the library name of libmpicxx in the additionalLibs of the CFDEMcoupling ($CFDEM_ADD_LIBS_DIR/$CFDEM_ADD_LIBS_NAME).
It is very probable the MPT cxx library is called differently, e.g. libmpichcxx .

Cheers,
Josef

Compiling CFDEM with g++ instead of mpic++

Forums:

Hi Rahul,

LIGGGHTS with g++ and fPIC

Hi Rahul,

For full access including downloads and forums, please register