Change domain seeding of simulation box for better CPUs usage

Submitted by Matteo on Mon, 09/26/2016 - 11:13

Hello,

I would like to change how the CPUs are distributed along my simulation box. In particular I am interested in finding a way to change the type of seeding by which each CPU is allocated to a region of the simulation domain.

E.g. By using the command

processors 4 4 1

I will get 4 CPUs along x, 4 CPUs along y and 1 CPU along z. Each CPU will be allocated to a region of space given by:
domain length along x / CPUs along x
domain length along y / CPUs along y
domain length along z / CPUs along z

This seeding is linear.
What source file should I modify if I want to modify the seeding law? Say following a (1-x)^-1 law.

What I want to do is to allocate more CPUs in regions of my domain where I have more particles.

Thanks
Matteo

Forums:

Matteo | Thu, 09/29/2016 - 14:28

Any information would be appreciated. I think this modification would be very useful to speed up my simulation, since some regions of the simulation domain have a much higher particles density.

richti83's picture

richti83 | Sat, 10/01/2016 - 21:56

What you are looking for is called adaptive MPI load-balancing and is available via the premium version [1], see this video where the partitions are shown as planes/boxes:
http://www.dcs-computing.com/dem-simulation-hoppersilo-discharge
From my point of view this is nothing one can achieve with some simple changes in current code bc. insert-fixes, neighbour handling and ghost particle/wall-element handling is affected.

From my experience with the premium version the fix loadbalance can improve computing cost up to 20%.

[1] http://www.dcs-computing.com/premium-versions-liggghtsr-and-cfdemrcoupling

I'm not an associate of DCS GmbH and not a core developer of LIGGGHTS®
ResearchGate | Contact

Matteo | Mon, 10/03/2016 - 09:16

Hi richti83,

Thanks for your answer. Unfortunately, for some reasons, we won't be able to get the premium version for this project.

I thought that in LIGGGHTS-PUBLIC there was a piece of code where the CPUs were allocated in space and thus the boundaries of each CPU were defined. Following this, other functions for neighbour and ghost particle computation would have compared the position of the particles with the one of the boundaries to decide if a particle was passing from a CPU's sub-domain to another one.
In this case, I think it would have been possible to define, following other laws, the location of the boundaries of the CPUs. I am not thinking to change the location of the boundaries of the CPUs during the computation, I would like just to split up the domain at the beginning of the simulation defining sub domains of the CPUs with different volumes.

Would this be possible?