Best setting for "processors command px py pz" in HPC

Submitted by limone on Mon, 03/26/2018 - 11:25

Dear All,

I have a couple of questions about the "processors command px py pz" (http://lammps.sandia.gov/doc/processors.html).
I am running a CFDEM simulation of a fluidized bed with HPC and I am trying to understand which could be the best set up for the processors.

My geometry is essentially a cylinder at the top of a truncated cone (pls see attached "mesh_limone.jpeg"), with the following approx. dimensions:
- max diameter = 200 mm (x-and-y-axis)
- height = 400 mm (z-axis)

The HPC has several k-cores, but probably, now, the best is to use up to 500 cores for the initial settings/tests.
Each node has 24 cores, therefore the total number of processors = px*py*pz should be multiple of 24 (or close to).

Due to this geometry, where the approx. dimensions are max_x = max_y = 2*z , I was thinking to distribute the processors as in these examples (px = py = 2*pz):

(a) processor 3 3 6
(b) processor 4 4 8
(c) processor 5 5 10

My doubts/questions are:

(1) I am using the "scotch decomposition" and I was thinking if it is possible to have a non-linear grid for processor pz (in the z-axis). Is it possible such option? Since most of my particles will stay at the bottom half of my geometry I was thinking that more processors in the z-axis (i.e. pz) are needed at the bottom half than at the upper half of my geometry.

(2) By using one of the "processors keywords", as "grid", "onelevel", "twolevel", "numa" or "custom", can the HPC performance improve (maybe less computational time) ?

Thanks,
Limone

AttachmentSize
Image icon mesh_limone.jpeg152.27 KB