pull, compile, run CFD-DEM coupling src, solver and testcase

cgoniva's picture
Submitted by cgoniva on Sat, 11/06/2010 - 09:35

Dear all,

In order to ease your life I summed up the commands necessary to pull, compile, run CFD-DEM coupling src, solver and testcase:

#- this was tested with the latest version of liggghts and OpenFOAM(R)-1.6

#- go to the directory where you want the files to live

#- to get the files from GIT use the command:
git clone git://cfdem.git.sourceforge.net/gitroot/cfdem/cfdem mycfdem

#- you'll get a directory:
mycfdem

#- this directory includes the directories:
cfdemParticle cfdemSolverPisoExplicit settlingTest

#####################################
# compile src
#####################################

#- change the path and type:
cd mycfdem
cd cfdemParticle

#- to compile the src type:
rmdepall
wclean
wmake libso

#- you should then get a message like:
'/home/XXX/OpenFOAM/XXX-1.6/lib/linuxGccDPOpt/liblagrangianCFDEM_shared.so' is up to date.

#####################################
# compile solver
#####################################

#- change the path and type:
cd ../cfdemSolverPisoExplicit/

#- to compile the solver type:
rmdepall
wclean
wmake

#####################################
# run testcase
#####################################

#- change the path and type:
cd ../settlingTest/

#- make sure the path of your pizza installation is set correctly in your Allrun.sh file,
#- for my installation it is: pizzaPath="$HOME/LAMMPS/pizza-16Mar09/src"
#- to you find the correct path for you type:
which pizza

#- make sure you use the soft link for the liggghts command (instead of the entry in the .bshrc),
#- otherwise the Allrun.sh script will not work properly

#- to run the testcase type:
./Allrun.sh

Forums:

phsieh2005 | Sat, 11/06/2010 - 13:04

Hi, Chris,

Thanks for the detailed compiling steps.

I found out that the problem I had previously was due to the fact that I am using OpenFOAM-1.7.x. The turbulence model is handled differently. So, I had to edit options to address this issue. However, I encountered a new problem, which I am not sure if it is OpenFOAM-1.7.x related or cfdem related. The compiling process cannot find "sys/stat.h" in twoWayFiles.H. I actually cannot find sys/stat.h anywher including OpenFOAM-1.7.x.

Pei-Ying
------------------------------------
phsieh@rutgers:~/cfdem/mycfdem/cfdemParticle> wmake libso
wmakeLnInclude: linking include files to ./lnInclude
Making dependency list for source file cfdemCloud/cfdemCloud.C
Making dependency list for source file subModels/forceModel/forceModel/forceModel.C
Making dependency list for source file subModels/forceModel/forceModel/newForceModel.C
Making dependency list for source file subModels/forceModel/DiFeliceDrag/DiFeliceDrag.C
Making dependency list for source file subModels/voidFractionModel/voidFractionModel/voidFractionModel.C
Making dependency list for source file subModels/voidFractionModel/voidFractionModel/newVoidFractionModel.C
Making dependency list for source file subModels/voidFractionModel/centreVoidFraction/centreVoidFraction.C
Making dependency list for source file subModels/locateModel/locateModel/locateModel.C
Making dependency list for source file subModels/locateModel/locateModel/newLocateModel.C
Making dependency list for source file subModels/locateModel/standardSearch/standardSearch.C
Making dependency list for source file subModels/meshMotionModel/meshMotionModel/meshMotionModel.C
Making dependency list for source file subModels/meshMotionModel/meshMotionModel/newMeshMotionModel.C
Making dependency list for source file subModels/meshMotionModel/noMeshMotion/noMeshMotion.C
Making dependency list for source file subModels/momCoupleModel/momCoupleModel/momCoupleModel.C
Making dependency list for source file subModels/momCoupleModel/momCoupleModel/newMomCoupleModel.C
Making dependency list for source file subModels/momCoupleModel/explicitCouple/explicitCouple.C
Making dependency list for source file subModels/regionModel/regionModel/regionModel.C
Making dependency list for source file subModels/regionModel/regionModel/newRegionModel.C
Making dependency list for source file subModels/regionModel/allRegion/allRegion.C
Making dependency list for source file subModels/dataExchangeModel/dataExchangeModel/dataExchangeModel.C
Making dependency list for source file subModels/dataExchangeModel/dataExchangeModel/newDataExchangeModel.C
could not open file sys/stat.h for source file subModels/dataExchangeModel/dataExchangeModel/newDataExchangeModel.C
Making dependency list for source file subModels/dataExchangeModel/twoWayFiles/twoWayFiles.C
could not open file sys/stat.h for source file subModels/dataExchangeModel/twoWayFiles/twoWayFiles.C
Making dependency list for source file subModels/averagingModel/averagingModel/averagingModel.C
Making dependency list for source file subModels/averagingModel/averagingModel/newAveragingModel.C
Making dependency list for source file subModels/averagingModel/dilute/dilute.C
Making dependency list for source file subModels/averagingModel/dense/dense.C
SOURCE=cfdemCloud/cfdemCloud.C ; g++ -m64 -Dlinux64 -DWM_DP -Wall -Wextra -Wno-unused-parameter -Wold-style-cast -Wnon-virtual-dtor -O3 -DNoRepository -ftemplate-depth-40 -I ./cfdemParticle -I/home/phsieh/OpenFOAM/OpenFOAM-1.7.x/src/finiteVolume/lnInclude -I/home/phsieh/OpenFOAM/OpenFOAM-1.7.x/src/transportModels -I/home/phsieh/OpenFOAM/OpenFOAM-1.7.x/src/transportModels/incompressible/lnInclude -I/home/phsieh/OpenFOAM/OpenFOAM-1.7.x/src/turbulenceModels/incompressible/turbulenceModel -I/home/phsieh/OpenFOAM/OpenFOAM-1.7.x/src/meshTools/lnInclude -I/home/phsieh/OpenFOAM/OpenFOAM-1.7.x/src/lagrangian/basic/lnInclude -IlnInclude -I. -I/home/phsieh/OpenFOAM/OpenFOAM-1.7.x/src/OpenFOAM/lnInclude -I/home/phsieh/OpenFOAM/OpenFOAM-1.7.x/src/OSspecific/POSIX/lnInclude -fPIC -c $SOURCE -o Make/linux64GccDPOpt/cfdemCloud.o

phsieh2005 | Mon, 11/08/2010 - 12:51

Hi, Christoph,

I found several stat.h on my OpenSuSE system. Maybe I will try to include it on my PATH.

Also, how does cfdem find liggghts stuffs? Do I need to include liggghts in my PATH too?

Thanks!

Pei-Ying

cgoniva's picture

cgoniva | Tue, 11/09/2010 - 14:32

Hi!

It should not be necessary to include the liggghts path anywhere in the CFD-DEM coupling.
They communicate via files and both programs know their locations.

Note: in order to use the *.sh scripts you need to use a symbolic link the loggghts command.
(see the liggghts installation tutorial)

Cheers,Chris

phsieh2005 | Thu, 11/11/2010 - 01:02

Hi, Chris,

I am getting errors when running the Allrun.sh script due to konsole. I am using OpenSUSE KDE, so, in the Allrun.sh file, I switched from gnome-terminal to konsole. The errors are:

konsole(12931)/kdeui (kdelibs): Attempt to use QAction "change-profile" with KXMLGUIFactory!
konsole(12940)/kdeui (kdelibs): Attempt to use QAction "change-profile" with KXMLGUIFactory!
simulation finisehd? ...press enter to proceed

However, when I manually entered the CFDrun.sh and DEMrun.sh commands into their terminals, the case ran. I am not sure if the results are correct or now. In paraview, I saw only 1 particle falling, then, bounces up.

Pei-Ying

cgoniva's picture

cgoniva | Thu, 11/11/2010 - 07:47

Hi Pei-Ying,

I'm happy to hear that the testcase showed only one particle falling and bouncing up again - that's exactly what it should do!!! :-) Seems you are now ready to start working with CFD-DEM coupling :-)

As you wrote, the scripts are not necessary - you could type the commands by hand - but they can really ease your life. If you manage to fix the script to get it working for your OpenSUSE KDE it would be great if you could contribute it here.

Cheers,
Christoph

tshan | Sat, 11/20/2010 - 18:48

Hi, Chris,

When I ran wmake in /cfdemSolverPisoExplicit/, there were some errors:

...
cfdemSolverPisoExplicit.C:45:24: error: cfdemCloud.H: No such file or directory
cfdemSolverPisoExplicit.C:46:28: error: explicitCouple.H: No such file or directory
cfdemSolverPisoExplicit.C: In function ‘int main(int, char**)’:
cfdemSolverPisoExplicit.C:60: error: ‘cfdemCloud’ was not declared in this scope
cfdemSolverPisoExplicit.C:60: error: expected ‘;’ before ‘particleCloud’
cfdemSolverPisoExplicit.C:75: error: ‘particleCloud’ was not declared in this scope
/opt/openfoam171/src/finiteVolume/lnInclude/readPISOControls.H:11: warning: unused variable ‘transonic’
/opt/openfoam171/src/finiteVolume/lnInclude/readPISOControls.H:14: warning: unused variable ‘nOuterCorr’
make: *** [Make/linux64GccDPOpt/cfdemSolverPisoExplicit.o] Error 1

It could not find cfdemCloud.H and explicitCouple.H, how can I resolve this issue? B.t.w., I use openFoam 1.7.1 and Ubuntu 10.04.

Thanks for your reply.

Best,
Tong

WeiJiSeniorDesign | Sun, 11/21/2010 - 18:21

I had this same problem. You have to add the path to those missing files in the Make/options file. For me, this looked like

-I/home/chris/mycfdem/cfdemParticle/lnInclude

I also had problems with some OpenFOAM files. Here's the options file that worked for me on a 32bit system:

# 1 "options"
# 1 ""
# 1 ""
# 1 "options"
EXE_INC = \
-I$(LIB_SRC)/turbulenceModels/incompressible/turbulenceModel \
-I$(LIB_SRC)/transportModels \
-I$(LIB_SRC)/transportModels/incompressible/singlePhaseTransportModel \
-I$(LIB_SRC)/finiteVolume/lnInclude \
-I$(WM_PROJECT_USER_DIR)/src/lagrangian/cfdemParticle/lnInclude \
-I/home/chris/mycfdem/cfdemParticle/lnInclude \
-I/home/chris/OpenFOAM/OpenFOAM-1.6/lib/linuxGccDPOpt \
-L/home/chris/OpenFOAM/OpenFOAM-1.6/lib/linuxGccDPOpt/openmpi-1.3.3

EXE_LIBS = \
-L$(FOAM_USER_LIBBIN) \
-lincompressibleRASModels \
-lincompressibleLESModels \
-lincompressibleTransportModels \
-lfiniteVolume \
-llagrangianCFDEM_shared \
-lincompressibleTurbulenceModel \
-lPstream

I'd also take a second look at your cfdemParticle compilation. I had the same path problems with RASModel.h, but the compiler didn't stop with an error message. Pretty easy to miss. I had to change the line

-I$(LIB_SRC)/turbulenceModels/RAS/incompressible/RASModel

to

-I$(LIB_SRC)/turbulenceModels/incompressible/RAS/RASModel

I hope this helps!

-Chris

tshan | Mon, 11/22/2010 - 06:04

Hi, Chris,

There came to be another problem, which may have nothing to do with cfdem. When I conducted cfdemParticle compilation, another error occured:

...
averagingModel.C:(.text+0x272b): undefined reference to `Foam::GeometricField, Foam::fvPatchField, Foam::volMesh>::debug'
collect2: ld returned 1 exit status
make: *** [OpenFOAM.out] Error 1

I can not find the exact error message before, so just post the last several lines. These error even did not happen last week when I asked you about cfdemSolverPisoExplicit compilation.

Btw, with another problem, when I boot my CFD with ". /opt/openfoam171/etc/bashrc", there came out to be:

/usr/lib/gcc/x86_64-linux-gnu/4.4.3/../../../../lib/crt1.o: In function `_start':
(.text+0x20): undefined reference to `main'
collect2: ld returned 1 exit status

Does these two problems relate with each other? How can I solve the cfdemParticle compilation error?

Thanks for your kind reply.

Best,
Tong

WeiJiSeniorDesign | Mon, 11/22/2010 - 22:39

Tong,

The first error looks like the compiler can't find a library or a header file. Look a few lines above that error for a "file not found" or similar error. Find the file on your system, then add the path as a new line to your options file. You need to prefix it with a "-L" if it's a library or a "-I" if it's anything else. Also, if it's a library, add another line to the end of the file that has the library file name without the extension preceded by a "-l" (lowercase L).

I recommend reading the man page for gcc to help further understand what's going on. It helped me a lot while I was figuring out how to compile this.

The second error seems to be missing a file too, but I don't know why OpenFOAM would be doing that. It's not related to the above error. Did you download the binary or the source distribution? The 1.6 binary blob just worked on my system.

Good luck!

-Chris

msbentley's picture

msbentley | Wed, 11/24/2010 - 17:44

I had the same problem with the bashrc script using the OpenFOAM repo on ubuntu.

The problem occurs in the file $WM_PROJECT_DIR/etc/settings.sh, specifically in the openmpi setup:

export PINC="`mpicc --showme:compile`"
export PLIBS="`mpicc --showme:link`"

I had to replace these calls to mpicc with:

mpicc -compile-info and mpicc -link-info

Cheers, Mark

skyopener's picture

skyopener | Fri, 12/17/2010 - 09:05

Hello CFD-DEM developers,
thanks to your great work and also the detailed installing description.
the script doesn't work in my computer, so I run the CFD solver in the settingTest/CFD folder, and then run the liggghts solver in the settingTest/DEM folder. and it seems to be run correctly.
Then I added the calculation steps five times as before both the CFD and DEM.
the result is beyond expectation.
the particle falls down until it hits the wall and then it rebounds, and after a lift process, it falls again.
But after three hitting, the particle rebounds with a large velocity. and then the solver cracked.
To catch this problem, I turned off the CFD-DEM couple in the /settingTest/DEM/in.liggghts file. and then run the DEM solver alone. and the solver seems normal in the simulation. but the post-processing is cracked.
The content of the file:/settingTest/DEM/in.liggghts_init is:
----------------------------------------------------------------------------------------------------------------------------------------------------
#echo both

# Pour granular particles into chute container, then induce flow

atom_style granular
atom_modify map array
communicate single vel yes

boundary f f f
newton off

units si

region reg block 0 0.1 0 0.1 0 0.1 units box
create_box 1 reg

neighbor 0.003 bin
neigh_modify delay 0 binsize 0.01

#Material properties required for new pair styles

fix m1 all property/global youngsModulus peratomtype 5.e6
fix m2 all property/global poissonsRatio peratomtype 0.45
fix m3 all property/global coefficientRestitution peratomtypepair 1 0.3
fix m4 all property/global coefficientFriction peratomtypepair 1 0.5

#pair style
pair_style gran/hertz/history 1 0 #Hertzian without cohesion
pair_coeff * *

#timestep, gravity
timestep 0.00001
fix gravi all gravity 9.81 vector 0.0 -1.0 0.0

#walls
fix xwalls all wall/gran/hertz/history 1 0 xplane 0.0 0.1 1
fix ywalls all wall/gran/hertz/history 1 0 yplane 0 0.1 1
fix zwalls all wall/gran/hertz/history 1 0 zplane 0 0.01 1

#-import mesh from cad:
#fix cad1 all mesh/gran hopperGenauerSALOME.stl 1 1.0 0. 0. 0. 0. 180. 0.

#-use the imported mesh as granular wall
#fix bucket_wall all wall/gran/hertz/history 1 0 mesh/gran 1 cad1

#cfd coupling
#fix cfd all couple/cfd/force 100 file ../CFD/couplingFiles

#particle insertion

#- distributions for insertion using pour
#fix pts1 all particletemplate/sphere 1 atom_type 1 density constant 2500 radius constant 0.001
#fix pts2 all particletemplate/sphere 1 atom_type 1 density constant 2500 radius constant 0.002
#fix pdd1 all particledistribution/discrete 1. 2 pts1 0.3 pts2 0.7
#variable alphastart equal 0.05
#region bc block 0.01 0.09 0.05 0.09 0.001 0.009 units box
#fix ins all pour/dev/packing 1 distributiontemplate pdd1 vol ${alphastart} 200 region bc

#- create single partciles
create_atoms 1 single 0.05 0.04 0.05 units box
set group all diameter 0.0001 density 3000

variable vx equal vx[1]
variable vy equal vy[1]
variable vz equal vz[1]
variable time equal step*dt
fix extra all print 100 "${time} ${vx} ${vy} ${vz}" file post/velocity.txt title "%" screen no

#apply nve integration to all particles that are inserted as single particles
fix integr all nve/sphere

#screen output
compute 1 all erotate/sphere
thermo_style custom step atoms ke c_1 vol
thermo 1000
thermo_modify lost ignore norm no
compute_modify thermo_temp dynamic yes

#insert the first particles so that dump is not empty
#dump myDump all stl 1 post/dump.stl
run 1
dump dmp all custom 200 post/dump.liggghts_init id type type x y z ix iy iz vx vy vz fx fy fz omegax omegay omegaz radius
#undump myDump

#force : f_couple_cfd[0] f_couple_cfd[1] f_couple_cfd[2]
#node : f_couple_cfd[6]
#cell id : f_couple_cfd[7]

run 75000
#write_restart liggghts.restart
----------------------------------------------------------------------------------------------------------------------------------------------------
after the post-processing commands:
d=dump("dump.liggghts_init")
v=vtk(d)
v.manyGran()
an error occured with the following message:
> v.manyGran()
200 400 600 800 1000 1200 1400 1600 1800 2000 2200 2400 2600 2800 3000 3200 3400 3600 3800 4000 4200 4400 4600 4800 5000 5200 5400 5600 5800 6000 6200 6400 6600 6800 7000 7200 7400 7600 7800 8000 8200 8400 8600 8800 9000 9200 9400 9600 9800 10000 10200 10400 10600 10800 11000 11200 11400 11600 11800 12000 12200 12400 12600 12800 13000 13200 13400 13600 13800 14000 14200 14400 14600 14800 15000 15200 15400 15600 15800 16000 16200 16400 16600 16800 17000 17200 17400 17600 17800 18000 18200 18400 18600 18800 19000 19200 19400 19600 19800 20000 20200 20400 20600 20800 21000 21200 21400 21600 21800 22000 22200 22400 22600 22800 23000 23200 23400 23600 23800 24000 24200 24400 24600 24800 25000 25200 25400 25600 25800 26000 26200 26400 26600 26800 27000 27200 27400 27600 27800 28000 28200 28400 28600
Traceback (most recent call last):
File "", line 1, in
File "/home/skyopener/gran_pizza_17Aug10/src/vtk.py", line 194, in manyGran
particleGran(file,atoms,names,len(self.data.snaps[0].atoms[0]))
File "/home/skyopener/gran_pizza_17Aug10/src/vtk.py", line 323, in particleGran
print >>f,"POINTS %d float" % len(atoms)
TypeError: object of type 'NoneType' has no len()
>

although many efforts have been payed, the problem cann't been cracked.
can someone give me a hint?
thanks for your reply

S.L Yang

cgoniva's picture

cgoniva | Mon, 12/20/2010 - 18:21

Hi S.L. Yang!

Is it possible that the particle leaves the domain?

pizza is running into trouble when there are no particles...
You might see this in the log data or terminal respectively.

concerning the script, some changes might be necessary: e.g. using bash instead of sh.
Going through the script line by line should help you find the problem.

Cheers,
Chris

skyopener's picture

skyopener | Thu, 12/23/2010 - 15:19

hi Chris,
With your hint, the problem has been cracked.
It's the problem that the particle leaves the CFD calculation domain.
To the second one, some lines such as :foamToVTK:paraview: can work properly. The CFD solver and DEM solver didn't run, more effort will be payed on it.
thanks!
S.L Yang

nt86 | Thu, 12/30/2010 - 18:26

Hello,

I'm trying to run Allrun.sh with the latest pizza and have the following output:

nikita@linux:/opt/mycfdem/settlingTest> ./Allrun.sh
simulation finisehd? ...press enter to proceed

Pizza.py (1 Oct 2006), a toolkit written in Python
type ? for help, CTRL-D to quit
Loading tools ...
Executing file: ./pizzaScriptInit
reading dump file...
200 400 600 800 1000 1200 1400 1600 1800 2000 2200 2400 2600 2800 3000 3200 3400 3600 3800 4000 4200 4400 4600 4800 5000 5200 5400 5600 5800 6000 6200 6400 6600 6800 7000 7200 7400 7600 7800 8000 8200 8400 8600 8800 9000 9200 9400 9600 9800 10000 10200 10400 10600 10800 11000 11200 11400 11600 11800 12000 12200 12400 12600 12800 13000 13200 13400 13600 13800 14000 14200 14400 14600 14800 15000
read 75 snapshots
75 snapshots selected out of 75
assigned columns: id type x y z ix iy iz vx vy vz fx fy fz omegax omegay omegaz radius
200 400 600 800 1000 1200 1400 1600 1800 2000 2200 2400 2600 2800 3000 3200 3400 3600 3800 4000 4200 4400 4600 4800 5000 5200 5400 5600 5800 6000 6200 6400 6600 6800 7000 7200 7400 7600 7800 8000 8200 8400 8600 8800 9000 9200 9400 9600 9800 10000 10200 10400 10600 10800 11000 11200 11400 11600 11800 12000 12200 12400 12600 12800 13000 13200 13400 13600 13800 14000 14200 14400 14600 14800 15000
wrote 75 granular snapshots in VTK format
Traceback (most recent call last):
File "/opt/gran_pizza_17Aug10/src/pizza.py", line 400, in
execfile(fullfile)
File "./pizzaScriptInit", line 5, in
exit()
File "/usr/lib64/python2.6/site.py", line 340, in __call__
raise SystemExit(code)
SystemExit: None
>>>

I don't know, if it's supposed behavior, it seems like python waits for my input now. If I enter exit() then the script go further and in the end the Paraview window is opened with axes and nothing else, no domain and no particles.

I have openSuse 11.3 x64.

cgoniva's picture

cgoniva | Sun, 01/02/2011 - 20:54

Hi!

Seems like the exit command in the script does not work in your case....

Paraview is not supposed to show anything - you need to select what you want to be displayed.

Check the VTK directories for the postprocessing data.

Cheers chris

nt86 | Thu, 01/06/2011 - 17:15

Thank you.

I've read some documentation and was able to vizualize 1 particle and the walls.

Where I can read about the fix couple/cfd/force for liggghts? I guess that is the main feature that couples DEM with CFD.
Maybe you will also share some more examples like the ones that you have on CFD-DEM page.

cgoniva's picture

cgoniva | Sun, 01/09/2011 - 21:24

Hey!

Great that you had some success with the visualization!

The coupling is not based solely on the DEM or the CFD side - it is more a "cooperation". Each code is supposed to do those calculations it can do best.

For the fluid-particle forces please have a look at the code in subModels/forceModel/...

those drag forces are added to trajectory integration in the DEM calculation.

There will be some new examples and a new version of the coupling soon. Please have some patience :-)

cheers,
Chris

tshan | Sun, 01/09/2011 - 16:27

Hi, Chris,

When I ran ./Allrun.sh, there's something wrong in the new terminal:

/home/tong/Software/mycfdem/settlingTest/CFDrun.sh: 11: Bad substitution
/home/tong/Software/mycfdem/settlingTest/CFDrun.sh: 19: source: not found
rm: cannot remove `./CFD/0.*': No such file or directory
/home/tong/Software/mycfdem/settlingTest/CFDrun.sh: 25: CFDrun: not found
cd: 30: can't cd to octave
rm: cannot remove `cfdemSolverPiso_settlingTest.eps': No such file or directory
/home/tong/Software/mycfdem/settlingTest/CFDrun.sh: 36: octave: not found

** (evince:7841): WARNING **: Error stating file '/home/tong/Software/mycfdem/settlingTest/cfdemSolverPiso_settlingTest.eps': No such file or directory

** (evince:7841): WARNING **: Error setting file metadata: No such file or directory

Could you tell me how to solve this? I have found the new solver(/home/tong/OpenFOAM/tong-1.6/applications/bin/linux64GccDPOpt/cfdemSolverPisoExplicit) exists.

Thanks a lot!

Best,
Tong