Restarting LIGGGHTS with bonds

Submitted by tjleps on Mon, 02/26/2018 - 22:12

Richti,

Have you been able to restart your bonds package with a different number of processors than were initially in the simulation? I've been trying to make this work for a little while now, as both the bond forming process, as well as the mesh/tet region style seem to have trouble with spanning processors, but I need to run simulations with ~100k-1M particles, so running on a single processor is not feasible.

Thanks,
Thomas Leps

Forums:

richti83's picture

richti83 | Thu, 03/01/2018 - 16:47

As far as I remember restartwith a different number of processors has never been possible with this version of LIGGGHTS. When I think about it I'm sure it is not possible as the bondhistory and the bond-partner vector is a per-processor memory block - this means you can not change number of processors after bonds has been created.
What you can do (and what I did in the project we used bonds -3 years ago- create a lot of prototypes of your bonded objects in single core operation and than use read_data command to load the particles into the main simulation, bond them in main simulation. Where main simulation can be a mpirun and creating of prototypes a single processor operation (this avoids the gap inside the prototypes) ....
long story short: I spent half a year to get a big simulation running and a lot of coding has been necessary to get it work.
Ref: Katterfeld et al., "COUPLED DISCRETE ELEMENT AND MULTI-BODY SYSTEM SIMULATION FOR DETERMINATION OF THE PERFORMANCE OF PRIMARY CRUSHERS", CFDEM®project user meeting & workshop, At: Linz, Volume: 1
(search for it at researchgate ;-))

I'm not an associate of DCS GmbH and not a core developer of LIGGGHTS®
ResearchGate | Contact

tjleps | Fri, 03/02/2018 - 06:07

Hopefully I can get the bonds to work across processors then. I took a break to work on some other research and am just getting back into it, but I recall getting an intermittent error where atoms would say that too many bonds had been created when they tried to bond across processor boundaries. I'm working on getting some complicated geometries packed with particles right now, but I may have a few more questions for you once I get to the bonding part again. Your bonds package has been a life saver for my research, I've had my hands full writing a much simpler package that adds magnetization to atoms.

mschramm | Tue, 03/06/2018 - 06:56

There must have been a change because I read in my particles with fortran code to fill a drum with flexible fibers with 4 cores, let the fibers settle, then write a restart file.
I then read in this file on a single core to do multiple DOE runs.... I do have the bond code running on 3.7.

richti83's picture

richti83 | Tue, 03/06/2018 - 08:11

I did'nt change the code since over 2 years ;-). But it might be possible to go from #4 (or N) cores to #1 core as in this case proc 0 gets all the data. But I would wonder if it works the other way (1 core in pre-simulation and 128 (or N) in main-simulation).
As long as the number of processors is constant restart should work for bonds in any case.

I'm not an associate of DCS GmbH and not a core developer of LIGGGHTS®
ResearchGate | Contact

ceemrp | Wed, 03/14/2018 - 12:09

I successfully ran a case with 650k particles that were packed into many different regions defined using mesh/tet style. What I did was create particles and bonds using 1 processor, write a restart file, then continue with 24 processors.

Nathan | Fri, 03/30/2018 - 03:33

Hi mates,

It is interesting to see a discussion regarding bond in LIGGGHT here. I am aware that Ceemrp is running with very large number of particles, 500K to 1m. However long does it usually take for each run, for example 1s in simulation ? And can you give some details regarding your computer configuration.

Thanks,

Regards,

tjleps | Thu, 05/10/2018 - 20:54

I haven't had a chance to do much benchmarking on just bonds added, however it doesn't slow things down too much from what I can tell vs not running with bonds. I'm running a much higher overhead fix on top of the bonds so it's hard for me to tell how much is lost by bonds. The biggest loss from running with bonds is that you need to have a larger skin depth if you want long bonds to form. It occurs to me though that after restarting, which I've managed to do with read_data, the bonds can probably be longer than the new neighbor length after the bonds are all formed.