compiling CFDEM on 2.0.x

cgoniva's picture
Submitted by cgoniva on Tue, 06/28/2011 - 11:08

Dear all,

there have been some issues with compiling cfdem with version 2.0.0 and 2.0.x.

It was tested and worked with 2.0.x.

With 2.0.0 errors ( MPI::Win::Free() ) might occur.

Please use the version 2.0.x and not the pre-compiled versions.

I'll do another testing and let you know as soon as everything has compiled fine.

Cheers,
Chris

Forums:

cgoniva's picture

cgoniva | Tue, 06/28/2011 - 20:10

I did some changes - hopefully it now compiles also with 2.0.x !!!

Sorry for the mis-understanding - compiling with 2.0.0 gives the MPI::Win::Free() errors.

Cheers,
Chris

akimbrell | Wed, 07/13/2011 - 16:08

Are the CFDEM instructions compatible with OpenFOAM 2.0.x? Or must we use the 1.7.x version? I ask because I have already installed 1.6-ext and 2.0.x and I really would like to avoid recompiling another OpenFOAM version if possible.

cgoniva's picture

cgoniva | Thu, 07/14/2011 - 13:06

Hi!

For me it works for 1.6-ext and 2.0.x. The instructions might be more or less ok for these versions.

Cheers,
Chris

akimbrell | Thu, 07/14/2011 - 19:02

Excellent, I will give it a try.

akimbrell | Fri, 07/15/2011 - 02:06

I am getting the following 4 errors when compiling the CFDEM solver:
/raid/linux8/apps/OpenFOAM/OpenFOAM-2.0.x/platforms/linux64GccDPOpt/lib/liblagrangianCFDEM_shared.so: undefined reference to `ompi_mpi_cxx_op_intercept'
/raid/linux8/apps/OpenFOAM/OpenFOAM-2.0.x/platforms/linux64GccDPOpt/lib/liblagrangianCFDEM_shared.so: undefined reference to `MPI::Datatype::Free()'
/raid/linux8/apps/OpenFOAM/OpenFOAM-2.0.x/platforms/linux64GccDPOpt/lib/liblagrangianCFDEM_shared.so: undefined reference to `MPI::Comm::Comm()'
/raid/linux8/apps/OpenFOAM/OpenFOAM-2.0.x/platforms/linux64GccDPOpt/lib/liblagrangianCFDEM_shared.so: undefined reference to `MPI::Win::Free()'

I have compiled both OpenFOAM 2.0.x and LIGGGHTS with the same compiler, same MPI references. I don't understand what is missing from MPI as the library file (cfdemParticle) compiled just fine. Any thoughts?

ckloss's picture

ckloss | Fri, 07/15/2011 - 12:56

>>I have compiled both OpenFOAM 2.0.x and LIGGGHTS with the same compiler, same MPI references.
Are you 100% sure also for the cfdem lib and solver? Normally these 4 errors are caused by linking to different MPIs

cheers,
Christoph

akimbrell | Sat, 07/16/2011 - 01:31

I am using OpenFOAM 2.0.x and newest LIGGGHTS version, and I am fairly sure that they are linking the same MPI (we are installing on a raid system for a networked cluster, I have recently consolidated our OpenMPI installations). One thing I can think of is I originally built OpenFOAM with a different MPI and then just did a 'wclean all' to remove all of the built files - perhaps it missed something. I will redo the process on Monday and see if I can find the problem.

akimbrell | Tue, 07/26/2011 - 00:44

Chris,

After resolving the compiler issues I still encounter the same problem when building either solver - note my libso file compiles correctly. Here is the relevant portion of the error code:

g++ -m64 -Dlinux64 -DWM_DP -Wall -Wextra -Wno-unused-parameter -Wold-style-cast -Wnon-virtual-dtor -O3 -DNoRepository -ftemplate-depth-100 -I/raid/linux8/apps/OpenFOAM/OpenFOAM-2.0.x/src/turbulenceModels/incompressible/turbulenceModel -I/raid/linux8/apps/OpenFOAM/OpenFOAM-2.0.x/src/transportModels -I/raid/linux8/apps/OpenFOAM/OpenFOAM-2.0.x/src/transportModels/incompressible/singlePhaseTransportModel -I/raid/linux8/apps/OpenFOAM/OpenFOAM-2.0.x/src/finiteVolume/lnInclude -I/raid/linux8/apps/OpenFOAM/OpenFOAM-2.0.x/src/lagrangian/cfdemParticle/lnInclude -I/raid/linux8/apps/OpenFOAM/OpenFOAM-2.0.x/src/dynamicFvMesh/lnInclude -I/raid/linux8/apps/OpenFOAM/OpenFOAM-2.0.x/src/dynamicMesh/lnInclude -IlnInclude -I. -I/raid/linux8/apps/OpenFOAM/OpenFOAM-2.0.x/src/OpenFOAM/lnInclude -I/raid/linux8/apps/OpenFOAM/OpenFOAM-2.0.x/src/OSspecific/POSIX/lnInclude -fPIC -Xlinker --add-needed Make/linux64GccDPOpt/cfdemSolverIB.o -L/raid/linux8/apps/OpenFOAM/OpenFOAM-2.0.x/platforms/linux64GccDPOpt/lib \
-lincompressibleRASModels -lincompressibleLESModels -lincompressibleTransportModels -lfiniteVolume -ldynamicFvMesh -ldynamicMesh -llagrangianCFDEM_shared -lOpenFOAM -ldl -lm -o /raid/linux8/apps/OpenFOAM/OpenFOAM-2.0.x/platforms/linux64GccDPOpt/bin/cfdemSolverIB_shared
/raid/linux8/apps/OpenFOAM/OpenFOAM-2.0.x/platforms/linux64GccDPOpt/lib/liblagrangianCFDEM_shared.so: undefined reference to `ompi_mpi_cxx_op_intercept'
/raid/linux8/apps/OpenFOAM/OpenFOAM-2.0.x/platforms/linux64GccDPOpt/lib/liblagrangianCFDEM_shared.so: undefined reference to `MPI::Datatype::Free()'
/raid/linux8/apps/OpenFOAM/OpenFOAM-2.0.x/platforms/linux64GccDPOpt/lib/liblagrangianCFDEM_shared.so: undefined reference to `MPI::Comm::Comm()'
/raid/linux8/apps/OpenFOAM/OpenFOAM-2.0.x/platforms/linux64GccDPOpt/lib/liblagrangianCFDEM_shared.so: undefined reference to `MPI::Win::Free()'
collect2: ld returned 1 exit status
make: *** [/raid/linux8/apps/OpenFOAM/OpenFOAM-2.0.x/platforms/linux64GccDPOpt/bin/cfdemSolverIB_shared] Error 1

It appears that you are calling MPI functions within the cfdemParticle library, I looked at the Make/options and there is an MPI include there which I don't really understand since it is being built with wmake which should be linked to the libraries in the first place. Can you explain this? At the moment I am out of ideas, I have compiled this about 5 different ways and each time I come up with the same problem. I am 100% sure that the compiler is the same version and the same mpi settings are being used throughout. To be honest it would be helpful if you would update your instructions to the newest versions as well.

ckloss's picture

ckloss | Tue, 07/26/2011 - 18:46

>> To be honest it would be helpful if you would update your instructions to the newest versions as well.
It's _impossible_ to keep everything 100% up to date for all systems, compilers, OpenF**m versions etc...
If you made it work, feel free to update the installation instructions and send it to us, we will put it online then. That's how open source works. Thx!

Christoph

akimbrell | Tue, 07/26/2011 - 19:46

As many times as I have gone through the whole process I do believe I might do that.

It appears I have gotten the solvers to compile correctly. It had to do with the settings in Make/options - I had to add the mpi_cxx library to the EXE_LIBS and it compiled with no errors. Here is my Make/options file for one of the solvers:

EXE_INC = \
-I$(LIB_SRC)/turbulenceModels/incompressible/turbulenceModel \
-I$(LIB_SRC)/transportModels \
-I$(LIB_SRC)/transportModels/incompressible/singlePhaseTransportModel \
-I$(LIB_SRC)/finiteVolume/lnInclude \
-I$(CFDEM_SRC_DIR)/lnInclude \
-I$(LIB_SRC)/dynamicFvMesh/lnInclude \
-I$(LIB_SRC)/dynamicMesh/lnInclude \
-I$(MPI_ARCH_PATH)/include \
-I$(CFDEM_LIGGGHTS_SRC_DIR)

EXE_LIBS = \
-lincompressibleRASModels \
-lincompressibleLESModels \
-lincompressibleTransportModels \
-lfiniteVolume \
-ldynamicFvMesh \
-ldynamicMesh \
-L$(MPI_ARCH_PATH)/lib \
-lmpi_cxx \
-l$(CFDEM_LIB_NAME)

The next step will be to test and see if it actually works with one of the examples.

akimbrell | Tue, 07/26/2011 - 21:17

It seems to work, I ran one of the tutorial cases in parallel and got no segmentation faults or anything similar. I will write up some installation instructions in the next few days to explain how I solved the problems.

Also one of the things I modified was the variables in the Make files and setup files to reflect an installation of cfdemParticle in the OpenFOAM src directory ($FOAM_SRC as opposed to $FOAM_USER_* directories). We work on a raid cluster and it requires having a networked installation rather than installing in a local user directory. Would these files be of use?

cgoniva's picture

cgoniva | Wed, 05/02/2012 - 19:31

Dear all,

it should be ok to add:

-L$(MPI_ARCH_PATH)/lib \
-lmpi_cxx \

to cfdemParticle/Make/options instead of adding it to the solvers options files.

@ akimbrell: THX for tracking this bug!

Cheers,
Chris

akimbrell | Wed, 07/20/2011 - 15:54

I have found that while my workstation supports the gcc 4.5 compiler, the server machines on our cluster are older SUSE operating systems with glibc 2.4, which only supports up to the gcc 4.1.2 compiler. OpenFOAM unfortunately does not compile properly with this compiler. I have spoken with my sys admin to start the upgrade process and hopefully I can then compile LIGGGHTS and OpenFOAM with the same compiler. I will let you know once I am able to do this if the process completes successfully.

akimbrell | Sat, 07/16/2011 - 01:33

One other thing, which OpenMPI are you using? We are using v1.4.3, I know there is a development version out but this is the most recent stable release.

cgoniva's picture

cgoniva | Fri, 07/15/2011 - 14:55

Hi!

I'll have a look at that and let you know asap (probably there is a bug somewhere...).

did you compile with 2.0.0 or 2.0.x ?

Cheers,
Chris

cgoniva's picture

cgoniva | Fri, 07/15/2011 - 18:28

Hi!

I could not reproduce the error on my 2.0.x installation, but i got the same error when trying to compile with the precompiled 2.0.0 version.

Cheers,
Chris

cas9637 | Sat, 05/12/2012 - 02:36

Hello,

I was running into an error, but I think I got it. I just needed to look deeper into the forum...

This post has the solution:

node/680

Thanks!
CAS

heliqun | Tue, 07/19/2011 - 16:03

Dear All,
I am new here, and first of all, my problem is where to find the files to be downloaded.
Additionally, does it work in other Linux or even in Windows XP?

Thanks a lot!
Dr. Liqun He

ckloss's picture

ckloss | Tue, 07/19/2011 - 18:01

Hi Liqun,

>>I am new here, and first of all, my problem is where to find the files to be downloaded.
the instructions for downloading CFDEM are here: node/237

>>Additionally, does it work in other Linux or even in Windows XP?
It should run on all major Linux distributions. As for XP, I am sure there is a way to do it, but this is not officially supported by us

Cheers,
Christoph

heliqun | Wed, 07/20/2011 - 01:05

HI Christoph,

Thanks a lot!
I will be trying it soon.

Regrds,
Liqun