cfdemCloud:: local mesh has zero cells. Please check the mesh and the decomposition!

Submitted by limone on Thu, 03/15/2018 - 18:08

Dear All,

I am trying to run a CFDEM simulation (similar to the Ergun Test Case) on a HPC cluster with 200 to 4000 cores.
For less than 50 cores (I used 18 cores and 48 cores) everything works perfectly, but increasing the number of cores up to 200 I get an error from OpenFOAM:

[0] --> FOAM FATAL ERROR:
[0]
cfdemCloud:: local mesh has zero cells. Please check the mesh and the decomposition!

Any help or suggestion ? I need urgently to use an high number of cores for this simulation/project and this error is blocking me since a long while.

I do not know if it helps, but with the usual method to create a mesh, i.e.

cd /home/CFD
blockMesh
surfaceFeatureExtract
decomposePar
mpirun -np 4 snappyHexMesh -overwrite -parallel
reconstructParMesh -constant -fullMatch
decomposePar
mpirun -np 4 renumberMesh -overwrite -parallel

my system (cylinder at the top of a truncated cone) is split as cake slices for each processor employed.
For instance, if I have/use 4 processors my system is split into 4 slices for the computation (...and I get 4 "processor" folders).

My guess, maybe totally nonsense, is that the increase of the number of processors, results in so many and "thin" slices, that for those processors there are not enough cells for the computation. Too stupid idea ?

Looking forward to hearing from you!

Best,
Lemon

paul | Thu, 03/15/2018 - 19:55

How many cells do you have in each coordinate direction?
I believe OpenFOAMs complaint may be valid

Try using scotch decomposition. It takes care of the slicing while:
- every proc gets about the same number of cells
- proc boundary faces are minimized

limone | Fri, 03/16/2018 - 10:32

Many thanks Paul!

To be honest I do not know where to look to know the number of cells in each coordinate direction.....

I was not thinking about the scotch decomposition and it could be a great solution... I am trying it right now !! Super thanks for this hint!!

I will let you know if it is gonna work later...

cheers,
Lemon