Hi everyone!
Thank you to all that have helped me install LIGGGHTS and CFDEM recently on my personal laptop. Now I am trying to install the same on my university's cluster. Obviously being a lowly postgrad student I am not on the sudo'er list, therefore I am not able to install things via git etc, therefore the I'm not quite able to follow the instructions online for LIGGGHTS and CFDEM compilation. I am able to rsync anything onto my home directory however
So far I have installed OpenFOAM-4.x and it is able to runs its tutorial cases. I rsync'ed the LIGGGHTS-master.zip and CFDEM-master.zip files onto the cluster and unzipped them in their own directories as the instructions state. I ran the cfdemSysTest initially and all the dependancies were correctly found (except for C3P0 of course).
*********************************************
* C F D E M (R) c o u p l i n g *
* *
* by DCS Computing GmbH *
* www.dcs-computing.com *
*********************************************
*********************************
CFDEM(R)coupling system settings:
*********************************
CFDEM_VERSION=PUBLIC
couple to OF_VERSION=4.x
compile option=Opt
check if paths are set correctly
valid:yes critical:yes - $CFDEM_PROJECT_DIR = /home/z5100344/CFDEM/CFDEMcoupling-PUBLIC-4.x
valid:yes critical:no - $CFDEM_PROJECT_USER_DIR = /home/z5100344/CFDEM/z5100344-PUBLIC-4.x
valid:yes critical:yes - $CFDEM_SRC_DIR = /home/z5100344/CFDEM/CFDEMcoupling-PUBLIC-4.x/src
valid:yes critical:yes - $CFDEM_SOLVER_DIR = /home/z5100344/CFDEM/CFDEMcoupling-PUBLIC-4.x/applications/solvers
valid:yes critical:yes - $CFDEM_TUT_DIR = /home/z5100344/CFDEM/CFDEMcoupling-PUBLIC-4.x/tutorials
valid:yes critical:yes - $CFDEM_LIGGGHTS_SRC_DIR = /home/z5100344/LIGGGHTS/LIGGGHTS-PUBLIC/src
valid:yes critical:yes - $CFDEM_LIGGGHTS_LIB_PATH = /home/z5100344/LIGGGHTS/LIGGGHTS-PUBLIC/src
valid:yes critical:yes - $CFDEM_LPP_DIR = /home/z5100344/LIGGGHTS/lpp/src
valid:yes critical:yes - $CFDEM_ADD_LIBS_DIR = /home/z5100344/CFDEM/CFDEMcoupling-PUBLIC-4.x/src/lagrangian/cfdemParticle/etc/addLibs_universal
valid:yes critical:yes - $CFDEM_LIB_DIR = /home/z5100344/CFDEM/CFDEMcoupling-PUBLIC-4.x/platforms/linux64GccDPInt32Opt/lib
valid:yes critical:yes - $CFDEM_APP_DIR = /home/z5100344/CFDEM/CFDEMcoupling-PUBLIC-4.x/platforms/linux64GccDPInt32Opt/bin
valid:yes critical:no - $CFDEM_USER_LIB_DIR = /home/z5100344/CFDEM/z5100344-PUBLIC-4.x/platforms/linux64GccDPInt32Opt/lib
valid:yes critical:no - $CFDEM_USER_APP_DIR = /home/z5100344/CFDEM/z5100344-PUBLIC-4.x/platforms/linux64GccDPInt32Opt/bin
valid:yes critical:no - $CFDEM_TEST_HARNESS_PATH = /home/z5100344/CFDEM/z5100344-PUBLIC-4.x/log/logFilesCFDEM-PUBLIC-4.x
valid:NO critical:no - $C3PO_SRC_DIR = does not exist
library names
$CFDEM_LIGGGHTS_LIB_NAME = lmp_auto
$CFDEM_LIB_NAME = lagrangianCFDEM-PUBLIC-4.x
$LD_LIBRARY_PATH = /home/z5100344/CFDEM/z5100344-PUBLIC-4.x/platforms/linux64GccDPInt32Opt/lib:/home/z5100344/CFDEM/CFDEMcoupling-PUBLIC-4.x/platforms/linux64GccDPInt32Opt/lib:/home/z5100344/OpenFOAM/ThirdParty-4.x/platforms/linux64Gcc/boost_1_55_0/lib:/home/z5100344/OpenFOAM/ThirdParty-4.x/platforms/linux64Gcc/CGAL-4.8/lib:/home/z5100344/OpenFOAM/ThirdParty-4.x/platforms/linux64Gcc/gperftools-svn/lib:/home/z5100344/OpenFOAM/ThirdParty-4.x/platforms/linux64Gcc/ParaView-5.0.1/lib/paraview-5.0:/home/z5100344/OpenFOAM/OpenFOAM-4.x/platforms/linux64GccDPInt32Opt/lib/openmpi-system:/home/z5100344/OpenFOAM/ThirdParty-4.x/platforms/linux64GccDPInt32/lib/openmpi-system:/usr/lib64/openmpi/lib:/home/z5100344/OpenFOAM/ThirdParty-4.x/platforms/linux64/mpc-1.0.1/lib64:/home/z5100344/OpenFOAM/ThirdParty-4.x/platforms/linux64/mpfr-3.1.2/lib64:/home/z5100344/OpenFOAM/ThirdParty-4.x/platforms/linux64/gmp-5.1.2/lib64:/home/z5100344/OpenFOAM/ThirdParty-4.x/platforms/linux64/gcc-4.8.5/lib64:/home/z5100344/OpenFOAM/z5100344-4.x/platforms/linux64GccDPInt32Opt/lib:/home/z5100344/OpenFOAM/site/4.x/platforms/linux64GccDPInt32Opt/lib:/home/z5100344/OpenFOAM/OpenFOAM-4.x/platforms/linux64GccDPInt32Opt/lib:/home/z5100344/OpenFOAM/ThirdParty-4.x/platforms/linux64GccDPInt32/lib:/home/z5100344/OpenFOAM/OpenFOAM-4.x/platforms/linux64GccDPInt32Opt/lib/dummy:/opt/openmpi/lib:/opt/python/lib
$WM_NCOMPPROCS =
$WM_LABEL_SIZE = 32
Additional lib settings
CFDEM_ADD_LIBS_DIR/CFDEM_ADD_LIBS_NAME = /home/z5100344/CFDEM/CFDEMcoupling-PUBLIC-4.x/src/lagrangian/cfdemParticle/etc/addLibs_universal/additionalLibs_4.x
CFDEM_ADD_LIB_PATHS =
CFDEM_ADD_LIBS =
LIGGGHTS library link (created during compilation of CFDEM)
ls: cannot access /home/z5100344/CFDEM/CFDEMcoupling-PUBLIC-4.x/platforms/linux64GccDPInt32Opt/lib/liblmp*: No such file or directory
*******************
g++:
/home/z5100344/OpenFOAM/ThirdParty-4.x/platforms/linux64/gcc-4.8.5/bin/g++
g++ (OpenFOAM) 4.8.5
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
gcc:
/home/z5100344/OpenFOAM/ThirdParty-4.x/platforms/linux64/gcc-4.8.5/bin/gcc
gcc (OpenFOAM) 4.8.5
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
mpic++:
/usr/lib64/openmpi/bin/mpic++
g++ (OpenFOAM) 4.8.5
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
mpirun:
/usr/lib64/openmpi/bin/mpirun
mpirun (Open MPI) 1.8.1
Report bugs to http://www.open-mpi.org/community/help/
**********************
additional packages...
I then attempted to run cfdemCompLIG, however right away it was unable to find files and gives the following output:
copying patch files for LIGGGHTS if available
cp: cannot stat `/home/z5100344/CFDEM/CFDEMcoupling-PUBLIC-4.x/src/LIGGGHTSpatch/*': No such file or directory
===========================================
Compiling sub-libraries of LIGGGHTS now...
Please provide the libraries to be compiled in the //home/z5100344/CFDEM/CFDEMcoupling-PUBLIC-4.x/src/lagrangian/cfdemParticle/etc/library-liggghts-list.txt file.
Libraries must be in: /home/z5100344/LIGGGHTS/LIGGGHTS-PUBLIC/src/../lib, or a path defined by the Line in the above file.
===========================================
deactivating all possible packages of LIGGGHTS now...
Please provide the packages to be compiled in the //home/z5100344/CFDEM/CFDEMcoupling-PUBLIC-4.x/src/lagrangian/cfdemParticle/etc/package-undo-liggghts-list.txt file.
Packages must be in: /home/z5100344/LIGGGHTS/LIGGGHTS-PUBLIC/src/../lib.
will change path...
/home/z5100344/LIGGGHTS/LIGGGHTS-PUBLIC/src
rm: cannot remove `log_compilePOEMSlib': No such file or directory
Uninstalling package POEMS
will change path...
/home/z5100344/LIGGGHTS/LIGGGHTS-PUBLIC/src
rm: cannot remove `log_compileDIPOLElib': No such file or directory
Package DIPOLE does not exist
will change path...
/home/z5100344/LIGGGHTS/LIGGGHTS-PUBLIC/src
rm: cannot remove `log_compileASPHERElib': No such file or directory
Uninstalling package ASPHERE
will change path...
/home/z5100344/LIGGGHTS/LIGGGHTS-PUBLIC/src
rm: cannot remove `log_compilePASCALlib': No such file or directory
Uninstalling package PASCAL
===========================================
activating packages of LIGGGHTS now...
Please provide the packages to be compiled in the //home/z5100344/CFDEM/CFDEMcoupling-PUBLIC-4.x/src/lagrangian/cfdemParticle/etc/package-liggghts-list.txt file.
Packages must be in: /home/z5100344/LIGGGHTS/LIGGGHTS-PUBLIC/src/../lib.
rm: cannot remove `/home/z5100344/CFDEM/CFDEMcoupling-PUBLIC-4.x/src/lagrangian/cfdemParticle/etc/log/log_compileLIGGGHTS': No such file or directory
// log_compileLIGGGHTS-2017-07-24-15:02 //
/home/z5100344/LIGGGHTS/LIGGGHTS-PUBLIC/src
rm: cannot remove `/home/z5100344/LIGGGHTS/LIGGGHTS-PUBLIC/src/lmp_auto': No such file or directory
rm -rf Obj_auto MAKE/Makefile.auto.options
rm: cannot remove `/home/z5100344/LIGGGHTS/LIGGGHTS-PUBLIC/src/liblmp_auto.a': No such file or directory
cleaning LIGGGHTS
compiling LIGGGHTS on one CPU
mv: cannot stat `style_contact_model.tmp': No such file or directory
ls: cannot access /usr/include/vtk*: No such file or directory
make[1]: Entering directory `/home/z5100344/LIGGGHTS/LIGGGHTS-PUBLIC/src/Obj_auto'
Makefile:468: *** Could not obtain VTK_MAJOR_VERSION. Stop.
make[1]: Leaving directory `/home/z5100344/LIGGGHTS/LIGGGHTS-PUBLIC/src/Obj_auto'
make: *** [auto] Error 2
ls: cannot access /usr/include/vtk*: No such file or directory
make[1]: Entering directory `/home/z5100344/LIGGGHTS/LIGGGHTS-PUBLIC/src/Obj_shlib_auto'
Makefile:468: *** Could not obtain VTK_MAJOR_VERSION. Stop.
make[1]: Leaving directory `/home/z5100344/LIGGGHTS/LIGGGHTS-PUBLIC/src/Obj_shlib_auto'
make: *** [auto] Error 2
I am not sure if there are things that are not being downloaded when I manually downloaded the .zip files from guthub and rsync'ed them to the cluster or if the make file is attempting to further download files and is unable to because of my limited permissions on the cluster.
I will of course ask the cluster admins for their advice also, however I'll expect their responses some time next decade...
Any help will be greatly appreciated,
Kind regards,
Rob
mardussa | Mon, 07/24/2017 - 07:13
Cluster Information
The cluster is UNSW's Leonardi cluster:
http://leonardi.unsw.wikispaces.net/home
It runs Rocks on top of CentOS.
One HP ProLiant DL385 G7 rack server head node
Seven HP BladeSystem C7000 blade enclosures, each containing eight full-height blade servers.
56 HP ProLiant BL685c G7 full-height blade server nodes with various configurations:
mpiexec (OpenRTE) 1.8.1
in case any of the information is of interest.
alice | Mon, 07/24/2017 - 09:08
Hello,
Hello,
the current issue seems to be one regarding vtk. There are a few possibilities: Either you are lucky, it is already installed somewhere and you can load it as a module. Otherwise, perhaps the admins can install it for you. Third option, and perhaps simplest one: you can install LIGGGHTS (and thus CFDEMcoupling) without vtk support. This can be done by changing the CFDEM_LIGGGHTS_MAKEFILE_NAME from auto to mpi in your .bashrc (or .cshrc). You'll then need lpp to convert the dump files to vtk, but at least you're gonna be able to run simulations.
Cheers,
Alice
mardussa | Mon, 07/31/2017 - 03:31
Hi Alice,
Hi Alice,
I gave that a go (changing from auto to mpi in my .bashrc) but still no luck. I ran make auto and make mpi from the /src directory and still no executable.
I also noticed that there is not actually a Makefile.mpi, but there is a Makefile.openmpi, so I tried by changing the .bashrc to openmpi and and also make openmpi but again, nothing except for the following output:
mv: cannot stat `style_contact_model.tmp': No such file or directory
make[1]: Entering directory `/home/z5100344/LIGGGHTS/LIGGGHTS-PUBLIC/src/Obj_openmpi'
mpic++ -O2 -funroll-loops -fstrict-aliasing -Wall -W -Wno-uninitialized -DLAMMPS_GZIP -DFFT_FFTW3 -c ../universe.cpp
In file included from ../fix_property_global.h:53:0,
from ../property_registry.h:51,
from ../force.h:56,
from ../universe.cpp:60:
../fix.h:221:16: warning: unused parameter ‘oldstep’ [-Wunused-parameter]
virtual void reset_timestep(bigint newstep,bigint oldstep)
^
mpic++ -O angle.o angle_hybrid.o atom.o atom_map.o atom_vec_atomic.o atom_vec_charge.o atom_vec.o atom_vec_ellipsoid.o atom_vec_hybrid.o atom_vec_line.o atom_vec_sph.o atom_vec_sphere.o atom_vec_sphere_w.o atom_vec_sph_var.o bond.o bond_hybrid.o bounding_box.o cfd_datacoupling.o cfd_datacoupling_file.o cfd_datacoupling_mpi.o cfd_regionmodel_none.o change_box.o citeme.o comm.o compute_atom_molecule.o compute_bond_local.o compute_centro_atom.o compute_cluster_atom.o compute_cna_atom.o compute_com.o compute_com_molecule.o compute_contact_atom.o compute_coord_atom.o compute.o compute_displace_atom.o compute_erotate_multisphere.o compute_erotate_sphere_atom.o compute_erotate_sphere.o compute_group_group.o compute_gyration.o compute_gyration_molecule.o compute_inertia_molecule.o compute_ke_atom.o compute_ke.o compute_ke_multisphere.o compute_msd.o compute_msd_molecule.o compute_nparticles_tracer_region.o compute_pair_gran_local.o compute_pe_atom.o compute_pe.o compute_pressure.o compute_property_atom.o compute_property_local.o compute_property_molecule.o compute_rdf.o compute_reduce.o compute_reduce_region.o compute_rigid.o compute_slice.o compute_stress_atom.o compute_surface.o compute_temp.o contact_models.o container_base.o create_atoms.o create_box.o custom_value_tracker.o delete_atoms.o delete_bonds.o dihedral.o dihedral_hybrid.o displace_atoms.o domain.o dump_atom_vtk.o dump.o dump_custom.o dump_custom_vtk.o dump_custom_vtm.o dump_decomposition_vtk.o dump_euler_vtk.o dump_image.o dump_local.o dump_local_gran.o dump_local_gran_vtk.o dump_mesh.o dump_mesh_stl.o dump_mesh_vtk.o dump_movie.o dump_particle.o dump_xyz.o error.o finish.o fix_adapt.o fix_addforce.o fix_ave_atom.o fix_ave_correlate.o fix_ave_euler.o fix_aveforce.o fix_ave_histo.o fix_ave_spatial.o fix_ave_time.o fix_base_liggghts.o fix_box_relax.o fix_buoyancy.o fix_cfd_coupling_convection.o fix_cfd_coupling_convection_impl.o fix_cfd_coupling_convection_species.o fix_cfd_coupling.o fix_cfd_coupling_force.o fix_cfd_coupling_force_implicit.o fix_check_timestep_gran.o fix_check_timestep_sph.o fix_contact_history.o fix_contact_history_mesh.o fix_contact_property_atom.o fix_contact_property_atom_wall.o fix_continuum_weighted.o fix.o fix_deform_check.o fix_deform.o fix_diam_max.o fix_drag.o fix_dt_reset.o fix_efield.o fix_enforce2d.o fix_external.o fix_fiber_spring_simple.o fix_freeze.o fix_gravity.o fix_heat_gran_conduction.o fix_heat_gran.o fix_insert.o fix_insert_pack.o fix_insert_rate_region.o fix_insert_stream.o fix_lb_coupling_onetoone.o fix_lineforce.o fix_massflow_mesh.o fix_massflow_mesh_sieve.o fix_mesh.o fix_mesh_surface.o fix_minimize.o fix_momentum.o fix_move.o fix_move_mesh.o fix_multicontact_halfspace.o fix_multisphere_break.o fix_multisphere_comm.o fix_multisphere.o fix_neighlist_mesh.o fix_nve.o fix_nve_limit.o fix_nve_noforce.o fix_nve_sph.o fix_nve_sphere.o fix_nve_sph_stationary.o fix_particledistribution_discrete.o fix_planeforce.o fix_print.o fix_property_atom.o fix_property_atom_timetracer.o fix_property_atom_tracer.o fix_property_atom_tracer_stream.o fix_property_global.o fix_read_restart.o fix_region_variable.o fix_respa.o fix_rigid.o fix_scalar_transport_equation.o fix_setforce.o fix_sph.o fix_sph_density_continuity.o fix_sph_density_corr.o fix_sph_density_summation.o fix_sph_pressure.o fix_spring.o fix_spring_rg.o fix_spring_self.o fix_store.o fix_store_force.o fix_store_state.o fix_template_multiplespheres.o fix_template_multisphere.o fix_template_sphere.o fix_viscous.o fix_wall.o fix_wall_gran.o fix_wall_region.o fix_wall_region_sph.o fix_wall_sph.o force.o global_properties.o granular_pair_style.o granular_styles.o granular_wall.o group.o image.o improper.o improper_hybrid.o input.o input_mesh_tet.o input_mesh_tri.o input_multisphere.o integrate.o irregular.o kspace.o lammps.o lattice.o library_cfd_coupling.o library.o main.o math_extra.o memory.o mesh_module.o mesh_module_liquidtransfer.o mesh_module_stress.o mesh_module_stress_servo.o mesh_mover.o min_cg.o min.o minimize.o min_linesearch.o modified_andrew.o modify.o modify_liggghts.o multisphere.o multisphere_parallel.o neigh_bond.o neighbor.o neigh_derive.o neigh_dummy.o neigh_full.o neigh_gran.o neigh_half_bin.o neigh_half_multi.o neigh_half_nsq.o neigh_list.o neigh_request.o neigh_respa.o neigh_stencil.o output.o pair.o pair_gran.o pair_gran_proxy.o pair_hybrid.o pair_hybrid_overlay.o pair_soft.o pair_sph_artvisc_tenscorr.o pair_sph.o particleToInsert.o particleToInsert_multisphere.o procmap.o properties.o property_registry.o random.o random_mars.o random_park.o read_data.o read_dump.o reader.o reader_native.o reader_xyz.o read_restart.o region_block.o region_cone.o region.o region_cylinder.o region_intersect.o region_mesh_tet.o region_neighbor_list.o region_plane.o region_prism.o region_sphere.o region_union.o region_wedge.o replicate.o respa.o rotate.o run.o run_time.o set.o sort_buffer.o special.o tet_mesh.o thermo.o timer.o tri_mesh.o tri_mesh_planar.o universe.o update.o variable.o velocity.o verlet.o write_data.o write_dump.o write_restart.o -lfftw -lstdc++ -o ../lmp_openmpi
/home/z5100344/OpenFOAM/ThirdParty-4.x/platforms/linux64/gcc-4.8.5/lib/gcc/x86_64-unknown-linux-gnu/4.8.5/../../../../x86_64-unknown-linux-gnu/bin/ld: cannot find -lfftw
collect2: error: ld returned 1 exit status
make[1]: *** [../lmp_openmpi] Error 1
make[1]: Leaving directory `/home/z5100344/LIGGGHTS/LIGGGHTS-PUBLIC/src/Obj_openmpi'
make: *** [openmpi] Error 2
Regards,
Robert
j-kerbl | Mon, 07/31/2017 - 11:44
Hi Robert,
Hi Robert,
I guess you have LIGGGHTS-PUBLIC 3.6 installed?
We've changed the Makefile names from 3.6 to 3.7 (https://github.com/CFDEMproject/LIGGGHTS-PUBLIC/tree/master/src/MAKE vs https://github.com/CFDEMproject/LIGGGHTS-PUBLIC/tree/3.6.0/src/MAKE).
For 3.6 I suggest to use the fedora_fpic Makefile. It is a very general one without VTK support. If you do upgrade to 3.7 you should upgrade the Coupling to 3.7 as well. Also the Makefile.mpi in 3.7 has the VTK libraries inside, but simply modifying line 77-79 takes care of that. https://github.com/CFDEMproject/LIGGGHTS-PUBLIC/blob/master/src/MAKE/Makefile.mpi#L79
I'll keep my fingers crossed.
Cheers,
Josef
mardussa | Tue, 08/08/2017 - 09:20
Hi Josef,
Hi Josef,
Thanks for the advice, I had been installing 3.6 but have now since updated to 3.7 for both LIGGGHTS and CFDEM.
So I now have a Makefile.mpi which i didn't in 3.6, and I have removed the definitions for the VTK variables? at line 77-79, leaving them blank. Unfortunately it still seems to be looking for something to do with VTK, as I am getting an error with dump_atom_vtk.cpp:
z5100344@leonardi:~/.../LIGGGHTS-PUBLIC/src $ make mpi
Creating list of contact models completed.
make[1]: Entering directory `/home/z5100344/LIGGGHTS/LIGGGHTS-PUBLIC/src/Obj_mpi'
mpic++ -O2 -funroll-loops -fstrict-aliasing -Wall -Wno-unused-result -fPIC -DLAMMPS_GZIP -DLAMMPS_VTK -c dump_atom_vtk.cpp
In file included from dump_atom_vtk.h:49:0,
from dump_atom_vtk.cpp:40:
dump_vtk.h:48:29: fatal error: vtkSmartPointer.h: No such file or directory
#include
^
compilation terminated.
make[1]: *** [dump_atom_vtk.o] Error 1
make[1]: Leaving directory `/home/z5100344/LIGGGHTS/LIGGGHTS-PUBLIC/src/Obj_mpi'
make: *** [mpi] Error 2
The other thing I noticed with the latest version of CFDEM was that there was no longer a CFDEM-PUBLIC/platforms directory present, so when I ran the cfdemSysTest it was having an issue with that.
Thanks for your help so far,
kind regards,
Robert
mardussa | Tue, 08/08/2017 - 09:24
I assume it will also have a
I assume it will also have a problem with the other dump_*_vtk.cpp and .h files when it gets to them.
j-kerbl | Thu, 08/10/2017 - 14:10
Hi Robert,
Hi Robert,
you'll probably still have "-DLAMMPS_VTK" in line 37 of the Makefile. Removing this will trigger the code to leave out the VTK parts for the compilation.
Cheers,
Josef
mardussa | Mon, 08/14/2017 - 08:16
Thank you very much that did
Thank you very much that did it! LIGGGHTS compiled, and as far as I can tell the rest of the CFDEM compilation went well also, as I am able to run the ErgunTestMPI problem with no worries.