Dear CFDEM Community
I am using Unresolved CFDEM to simulate particle infiltration in bed. I am facing problem now to simulate the case for larger duration due to high memory requirements as number of particles increase as simulation progresses and eventually gets killed even on large Linux-clusters. I know that Unresolved CFDEM has limit in terms of number of particles (oder of ~10^7) but my simulation gets killed despite total number of particles are less than the limit specified above.
I am running the case on 8 nodes each containing 28 number of cores,so in total 224 number of processors. In the CFD side, i am using scotch method of decomposition, which automatically divides the CFD domain optimally. In the DEM side, I tried with different arrangements of processors in x, y and z direction (28 4 2, 28 2 4, 14 8 2, etc.) but no success in optimizing the parallalization. I wanted to know if there is a way to optimize my paralization both in CFD and DEM side, so the killing of job due to high memory requirement can be avoided. more specifically, i have some questions/thoughts as follows:
1. Is there any way to tell LIGGGHTS (DEM side) that do the decomposition as done in OpenFOAM (CFD side) so that both CFD and DEM domain calculations are done on same node and may be this will reduce communication load among processors?
2. I saw the processor command (https://www.cfdem.com/media/DEM/docu/processors.html) and found an option to divide the domain with two levels by specifying number of cores used and decomposition in each core. However, I am not sure if it can solve the problem. Any comments on that and may be some example how to use this function efficiently.
3. I also saw that instead of specifying the decomposition in x, y and z direction, I can use '*' and LIGGGHTS will devide the domain with some optimization algorithm (minimizing the surface-to-volume ratio) but I am not sure if the surface-to-volume optimization can solve my problem as I think the memory problem is more related to to too many particles (this too many calculations) on some of my processors. It would be great if LIGGGHTS could devide the domain by optimizing number of particles. Is comment on this?
Looking forward to hear it from your experiences and let me know if any other point that I missed to point out.