A distro-agnostic approach to installing OpenMPI, FFTW and LIGGGHTS

Submitted by fskmh on Wed, 07/21/2010 - 18:33


Big fat disclaimer:
I come from a Slackware background, but since I expect most LIGGGHTS users out there to be running RHEL/Scientific Linux/CentOS/Fedora, I have made two assumptions regarding the build variable used by automake/autoconf in OpenMPI and FFTW. Replacing "--build=x86_64-redhat-linux" with something suited to your distro should be trivial. (To find out what build variable you should use for your distro, execute the command "gcc -v" and look in the output for "--build=".)

Assumptions
(01) Global installation, i.e. as root, so remember the implicit "su -l".
(02) Installation on x86_64. It would work just as well on 32 bit, which would require a few trivial changes to the steps below. (This mostly requires removing the -m64 compiler flags where they occur below and changing any "lib64" reference to "lib", e.g. /usr/local/lib64 would become /usr/local/lib etc.)

OpenMPI - http://www.open-mpi.org/
OpenMPI is basically the next generation of LAM/MPI (http://www.lam-mpi.org/). It is an MPI-2 implementation maintained in a collaboration between academia and industry. While your Linux distro of choice may have OpenMPI in a repository somewhere, it is best to compile OpenMPI using the compiler it will serve as a wrapper for. (Just as one would do with a vendor-specific implementation of MPI.)

(01) Download the most recent OpenMPI tar.bz2 archive (openmpi-x.y.z.tar.bz2).
(02) Move/copy the tar.bz2 archive to /usr/local/src
(03) cd /usr/local/src
(04) tar xf openmpi-x.y.z.tar.bz2
(05) cd openmpi-x.y.z.tar.bz2
(06) touch openmpi.conf
(07) chmod a+x openmpi.conf
(08) vi openmpi.conf (paste the following code and save)

#!/bin/bash
#
# Configure wrapper for compiling OpenMPI from source with GNU or Intel
# compilers.
#
# Mathew Holleran, 20-07-2010

#************************************************************************
# Config stuff. You should probably not modify anything.
#************************************************************************
EXPECTED=openmpi
PRGNAM=`basename $PWD | awk -F- '{print $1}'`
VERSION=`basename $PWD | awk -F- '{print $2}'`

#************************************************************************
# Script functions
#************************************************************************

#************************************************************************
# Function sanity_check() : Check that script is run in the correct place
# and extract a version number with some error-checking.
#************************************************************************
sanity_check(){
if [ ${PRGNAM} != ${EXPECTED} ]; then
echo "!-> Expected top-level folder of ${EXPECTED} - bailing."
exit 1
else
# Test for empty string
if [ -z ${VERSION} ]; then
echo "!-> Could not parse version number."
exit 1
else
# Test for something other than leading digit in string
if echo ${VERSION} | grep -qv '^[0-9]'; then
echo "!-> Invalid version number - bailing."
exit 1
fi
fi
fi
}

#************************************************************************
# Function config() : Assemble configuration parameters for building.
#************************************************************************
config(){
export RSHCOMMAND="ssh -q"
if [ ${CC} == "icc" ]; then
PREFIX="/usr/local/openmpi-${VERSION}-intel"
else
PREFIX="/usr/local/openmpi-${VERSION}"
fi
./configure --prefix=${PREFIX} \
--with-ft=LAM --with-mpi-f90-size=large --mandir=${PREFIX}/man \
--disable-mca-dso --enable-shared --enable-static \
--enable-smp-locks --enable-mpi-threads --disable-ipv6 \
--with-slurm --with-sge --build=x86_64-redhat-linux
}

#************************************************************************
# Function report() : Print summary of compiler configuration.
#************************************************************************
report(){
echo
echo "I-> Configuration completed:"
echo "I-> RSHCOMMAND=${RSHCOMMAND}"
echo "I-> prefix=${PREFIX}"
echo "I-> CC=${CC}, CXX=${CXX}"
echo "I-> FC=${FC}, C90=${F90}"
}

#************************************************************************
# Script begins. Compiler variables can be manually below.
#************************************************************************
case $1 in
'gnu')
sanity_check
export LC_ALL="C"
export CC=gcc
export CFLAGS=-m64
export CXX=g++
export CXXFLAGS=-m64
export FC=gfortran
export FFLAGS=-m64
export F90=gfortran
export FCFLAGS=-m64
PREFIX="/usr/local/openmp-${VERSION}"
config
report
;;
'intel')
sanity_check
# export LC_ALL="C"
export CC=icc
export CXX=icpc
export FC=ifort
export F90=ifort
PREFIX="/usr/local/openmp-${VERSION}-intel"
config
report
;;
*)
sanity_check
echo "I-> ${PRGNAM}-${VERSION}"
echo "!-> Additional parameter required: {intel|gnu}"
exit 1
;;
esac

exit 0

(09) ./openmpi.conf gnu
(10) If everything went well with the configuration you should have an autoconf generated Makefile. Now type "make && make install" (without the quotes) and hope for the best.
(11) cd /etc/profile.d
(12) touch openmpi.sh
(13) chmod a+x openmpi.sh
(14) vi openmpi.sh (paste the following code and save)

#!/bin/bash
VER="1.4.2"
if [ -d /usr/local/openmpi-${VER} ]; then
TEST=`which ifort`
if [ ! -z ${TEST} ]; then
export MPIPATH=/usr/local/openmpi-${VER}-intel
export LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:${MPIPATH}/lib"
export PATH="${MPIPATH}/bin:${PATH}"
export MANPATH="${MPIPATH}/man:${MANPATH}"
else
export MPIPATH=/usr/local/openmpi-${VER}
export LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:${MPIPATH}/lib"
export PATH="${MPIPATH}/bin:${PATH}"
export MANPATH="${MPIPATH}/man:${MANPATH}"
fi
fi

(15) EDIT THE ABOVE SO THAT $VER IS THE SAME AS x.y.z IN openmpi-x.y.z.tar.bz2

FFTW - http://www.fftw.org/
It appears that FFTW2 is being neglected by packagers in favour of FFTW3, so installing it manually should not cause any dependency-checking breakage. If the readers out there have any doubts they can install it from their respective distro repository.

(01) Download fftw-2.1.5.tar.gz
(02) Move/copy it to /usr/local/src
(03) cd /usr/local/src
(04) tar xf fftw-2.1.5.tar.gz
(05) cd fftw-2.1.5
(06) touch fftw.conf
(07) chmod a+x fftw.conf
(08) vi fftw.conf (paste the following code and save)

#!/bin/bash
export CFLAGS="-O2 -fPIC -m64"
export CXXFLAGS="-O2 -fPIC -m64"
./configure --prefix=/usr/local -libdir=/usr/local/lib64 \
--disable-static --enable-shared --enable-pentium-timer \
--enable-mpi --enable-threads \
--build=x86_64-redhat-linux

(09) ./fftw.conf
(10) make && make install
(11) cd /etc/ld.so.conf.d
(12) touch fftw-x86_64.conf
(13) vi fftw-x86_64.conf (paste just the one line below and save)

/usr/local/lib64

(14) Note that this library path matches the argument in fftw.conf.
(15) ldconfig -v

LIGGGHTS
(01) Download the newest version of LIGGGHTS and rename the file to a tar.gz as per the instructions on this site, e.g. liggghts_1p1p5.tar.gz
(02) At this point the user can exit back to their normal user account or continue as the root user.
(03) Copy/move the archive to /usr/local/src in the case of a global installation as root, or somewhere in your home directory as a user.
(04) tar xf liggghts_1p1p5.tar.gz
(05) cd liggghts_1p1p5/src
(06) make openmpi
(07) This should result in the building of the lmp_openmpi binary.
(08) On a dual-core PC it can be tested like this:

mpirun -np 3 lmp_openmpi

(09) Open a terminal/xterm and type "top", then press 1 to view the stats per core.

AttachmentSize
Plain text icon openmpi.conf_.txt3.16 KB
Plain text icon fftw.conf_.txt253 bytes
Plain text icon openmpi.sh_.txt512 bytes
Plain text icon fftw-x86_64.conf_.txt17 bytes