Hi Christoph,
Following your reply in a post "how to speed up DEM"
I have taken a look at my timing statistics, and it seems the "other time" takes up the most of the time consumed.
Pair time (%) = 312.239 (32.578)
Neigh time (%) = 9.10633 (0.950124)
Comm time (%) = 16.8501 (1.75809)
Outpt time (%) = 0.128907 (0.0134498)
Other time (%) = 620.111 (64.7004)
Do you have any idea what is the "other time", How do I reduce it?
My simulation can run, but at an very slow speed. The tricky thing is, my script is based on other's script, but his runs very fast, even with large number of particles (14 K). I only changed the parameters, including particle size, skin distance etc.
I am now trying to speed up my simulation, and I suspect it has something to do with my parameter settings.
Christoph, would you mind taking a look at my script?
Thanks in advance!
=====================
units si
atom_style granular
atom_modify map array
boundary f f f
newton off
region reg block -2.0 2.0 -2.0 3.0 -2 4 units box
create_box 3 reg
communicate single vel yes
neighbor 0.008 bin
neigh_modify delay 0
#Material properties required for new pair styles
fix m1 all property/global youngsModulus peratomtype 8.e10 8.e9 8.e10
fix m2 all property/global poissonsRatio peratomtype 0.3 0.3 0.3
fix m3 all property/global coefficientRestitution peratomtypepair 3 &
0.2 0.3 0.05 &
0.3 0.3 0.3 &
0.05 0.3 0.2
fix m4 all property/global coefficientFriction peratomtypepair 3 &
0.0 0.0 0.0 &
0.0 0.0 0.0 &
0.0 0.0 0.0
fix m5 all property/global characteristicVelocity scalar 2.
# type 1 : walls of boxex
# type 2 : filled particles
# type 3 : pariticle-wall
#New pair style
pair_style gran/hertz/history 1 0 #Hertzian without cohesion
pair_coeff * *
timestep 3e-6
#granular walls
fix upper all mesh/gran/stressanalysis upper_3.stl 1 1 0. 0. 0. 0. 0. 0.
fix lower all mesh/gran/stressanalysis lower_3.stl 1 1 0. 0. 0. 0. 0. 0.
fix wall all wall/gran/hertz/history 1 0 mesh/gran 2 upper lower
#define particles insertion into domain
fix pts1 all particletemplate/sphere 1 atom_type 2 density constant 3000 radius constant 0.0035
fix pts2 all particletemplate/sphere 1 atom_type 2 density constant 3000 radius constant 0.0025
fix pdd1 all particledistribution/discrete 1 2 pts1 0.8 pts2 0.2
#region and insertion
region bc block -0.08 0.08 -0.12 0.48 0 0.025 units box
group nve_group region reg
fix insert nve_group pour/dev/packing 1 distributiontemplate pdd1 vol 0.3 100 region bc
fix gravi all gravity 9.81 vector 0.0 -1.0 0.0
#apply nve integration to all particles that are inserted as single particles
fix NVE all nve/sphere
#output settings, include total thermal energy
compute 1 all erotate/sphere
thermo_style custom step atoms ke c_1 vol
thermo 50000
thermo_modify lost ignore norm no
compute_modify thermo_temp dynamic yes
#insert the first particles so that dump is not empty
run 1
dump dmp all custom 50000 post/dump.test_v_8_02 id type type x y z ix iy iz vx vy vz fx fy fz omegax omegay omegaz radius
run 100000
unfix insert
group exclude_pwall region bc
unfix NVE
fix NVE_new exclude_pwall nve/sphere
run 400000
#Apply wiggle, sample preparation
fix wigglecylin1 all move/mesh/gran wiggle -0.003 0 0 0.08 upper
fix wigglecylin2 all move/mesh/gran wiggle -0.003 0 0 0.08 lower
run 600000
unfix wigglecylin1
unfix wigglecylin2
run 200000
#==========write a restart file==========
#restart 850000 restartfile
#creat particle-wall
region pwall_region block -0.08 0.08 0.49 0.493 0 0.025 units box
lattice custom 0.004 a1 1 0 0 a2 0 1 0 a3 0 0 1 basis 0.5 0.5 0.5
create_atoms 3 region pwall_region
group pwall region pwall_region
set group pwall diameter 0.004 density 5000
fix setrigid pwall rigid single force * off on off torque * off off off
neigh_modify exclude type 3 3
# apply 0.196 N/600 = 0.2 N/particle confining pressure
unfix gravi
fix vertical pwall aveforce 0.0 -1 0
run 300000
unfix vertical
fix vertical_new pwall aveforce 0.0 -0.3 0
#Presure: 18 Kpa
unfix m4
fix m4_new all property/global coefficientFriction peratomtypepair 3 &
0.0 0.0 0.0 &
0.0 0.5 0.0 &
0.0 0.0 0.0
fix forceofmesh all ave/time 50 50 50000 f_lower[1] f_lower[2] f_lower[3] file Force_v_8_02.txt
dump dmp2 pwall custom 50000 post/dump.pwall_v_8_02 id type type x y z ix iy iz vx vy vz fx fy fz omegax omegay omegaz radius
fix shear all move/mesh/gran linear 0.0002 0 0 units box lower 1
run 50000000
ckloss | Wed, 03/28/2012 - 17:07
There are a couple of things
There are a couple of things which contribute to that, e.g. fix rigid in its current form is not fully parallel. Also, if you run in parallel, load imbalance can also contribute to other time. Giving you instructions how to speed-up this specific simulation is beyond the scope of this forum, so I'd invite you to take part in our next training session.
Cheers, Christoph
venes520 | Thu, 03/29/2012 - 04:56
Dear Christoph,Thanks you
Dear Christoph,
Thanks you for your reply!
Can you explain more on the fix rigid form? what do you mean by "not fully parallel', the simulation is already slow before I introduce the rigid body.
About the load imbalance, I actually tested my script in both parallel and single, it seems the total simulation time in parallel is shorter.
However, for single processor, the "other time" remains very high at 40% while the "pair time" is about 50%.
Do you have any idea why the "other time" is still so high using single processor?
Thanks very much!
venes520 | Thu, 03/29/2012 - 04:55
Hi Christoph, Other than the
Hi Christoph,
Other than the training session, is there any other way that I can request consulting service from you guys? The next session is at June, so late.
By the way,any instructions on running in parallel efficiently?
Thanks ~