bonding across processor boundaries

Submitted by tjleps on Sat, 03/10/2018 - 21:24

I know I've discussed this at some level before but, Richti, it sounds as though you've run large simulations with your bonds package with multiple processors? I seem to get intermittent errors when I try to, for example:

creating bond btw atoms 53 and 54 (i has now 4 bonds) at step 2
creating bond btw atoms 53 and 288960 (i has now 5 bonds) at step 2
creating bond btw atoms 53 and 288961 (i has now 6 bonds) at step 2
creating bond btw atoms 54 and 52 (i has now 1 bonds) at step 2
creating bond btw atoms 54 and 53 (i has now 2 bonds) at step 2
ERROR: Internal fix bond/create error (../fix_bond_create_gran.cpp:465)

Comment from the off: The error says: Try this again and I'll be back! (../fix_bond_create_gran.cpp:465)
-------------------------------------------------------
Primary job terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.

This sim has ~1M particles across 4 processors, and is successful when run on a single processor. Thanks for any support you can offer.

richti83's picture

richti83 | Sun, 03/11/2018 - 16:11

Let's have a look into the source code:

for(int jp = 0; jp < npartner[j]; jp++)
if(partner[j][jp] == tag[i]) found = 1;
if (!found) error->all(FLERR,"Internal fix bond/create error");

This means, another proc didn't find the same partner atoms. This often happens when there are more possible bonds as allowed by bonds per atom (controlled at create box command) or when the probebility is smaller than 1
As far as I remember I never successfull solved this and instead used bonds per atom very high (20 in my last case) and probebility=1
You should take care on the maximum bond distance and the neighbor cutoff distance, too.
To be honest I stoppped working on bond package 3 years ago as there is no recent project where we would use it.

I'm not an associate of DCS GmbH and not a core developer of LIGGGHTS®
ResearchGate | Contact

tjleps | Mon, 03/12/2018 - 04:43

Hmmm, that fix didn't change the behavior in my script, I think I'll need to make another workaround. Before I get too deep into the problem, from your memory of how the code is formatted, would it be relatively straight forward for me to add functionality that allowed a bond list to be read in from a file, such that I could run a simulation on a single proc, generate the bonds, then make a new simulation that simply read in the bond list created in the previous? I'm pretty invested in making this work, so I'm looking for the path of least resistance.

Thanks for your help,
Thomas

richti83's picture

richti83 | Mon, 03/12/2018 - 08:42

Hi,
this sounds like a good Idea and I would say that this functionality is already there from LAMMPS, see https://github.com/richti83/LIGGGHTS-WITH-BONDS/blob/679ce24504b1922e24d...
This calls https://github.com/richti83/LIGGGHTS-WITH-BONDS/blob/679ce24504b1922e24d...
I don't know if it works for multiple processors ..
You should only find out the correct file format, see this post for a hint (but there hybrid granular molecular ellipsoid is used ..)
https://www.cfdem.com/forums/bonded-particle-model-0

I'm not an associate of DCS GmbH and not a core developer of LIGGGHTS®
ResearchGate | Contact

tjleps | Tue, 03/13/2018 - 03:46

Great, I'll let you know how it goes. Is the article you sent me your preferred article for citing your work developing the LIGGGHTS with bonds package? I want to make sure you get some credit in a couple of journal papers I'm writing that use it heavily.

richti83's picture

richti83 | Wed, 03/14/2018 - 09:31

Hi,

yes the full paper of ATK Congress [1] and a presentation at 1st CFDEM(R)User Meeting and Workshop March 2016 [2] are the only publications about the bond feature from my side.

To say it clear: I did not invent the parallel bond [5] nor implemented it in liggghts(r), it bases on a fork of 'polyun' [3],[4], I just maintained it, as I needed it for the crusher project.

[1] https://www.researchgate.net/publication/308748622_Gekoppelte_Diskrete_E...
[2] https://www.researchgate.net/publication/308748518_COUPLED_DISCRETE_ELEM...
[3] https://github.com/Polyun/LIGGGHTS-PUBLIC
[4] http://tm.spbstu.ru/Implementation_of_V_model_in_LIGGGHTS
[5] https://www.sciencedirect.com/science/article/pii/S1365160904002874

I'm not an associate of DCS GmbH and not a core developer of LIGGGHTS®
ResearchGate | Contact

tjleps | Thu, 05/10/2018 - 21:07

You were correct that write_data and read_data were more or less already good to go, however for some reason it always writes molecule id as 0, which is fine because it can be changed in post. However, it's a bit of a pain for millions of particles, do you have any idea why this might be happening? I've even tried using set to make the molecule id 1 and it still writes as 0.