Setting up the parallel environment

In addition to the installation steps described in Section 2 (see page ) you just have to set the variable PARA_ARCH to MPI, i.e. in sh/bash/ksh syntax:

export PARA_ARCH=MPI

This will cause sysname to append the string _mpi to the system name and the scripts like jobex will take the parallel binaries by default. To call the parallel versions of the programs ridft, rdgrad, dscf, grad, ricc2, or mpgrad from your command line without explicit path, expand your $PATH environment variable to:

export PATH=$TURBODIR/bin/`sysname`:$PATH

The usual binaries are replaced now by scripts that prepare the input for a parallel run and start mpirun (or poe on IBM) automatically. The number of CPUs that shall be used can be chosen by setting the environment variable PARNODES:

export PARNODES=8

The default for PARNODES is 2.

Finally the user can set a default scratch directory that must be available on all nodes. Writing scratch files to local directories is highly recommended, otherwise the scratch files will be written over the network to the same directory where the input is located. The path to the local disk can be set with

export TURBOTMPDIR=/scratch/username/

This setting is automatically recognized by the parallel ridft and ricc2 programs. Note that this does not set the path for the integer scratch files for dscf (see section below about twoint of keyword $scfintunit).

If TURBOTMPDIR is not set by the user, ridft will check for a /tmp, /scr, or /work directory. If one of those is found, it will be used by default.

On all systems TURBOMOLE is using the MPI library that has been shipped with your operating system.

On Linux for PCs and Itanium2 systems HP-MPI is used -- see
http://www.hp.com/go/mpi/http://www.hp.com/go/mpi

The binaries that initialize MPI and start the parallel binaries (mpirun) are located in the $TURBODIR/mpirun_scripts/HPMPI/ directory.

Note: the parallel TURBOMOLE modules (except ricc2) need an extra server running in addition to the clients. This server is included in the parallel binaries and it will be started automatically--but this results in one additional task that usually does not need any CPU time. So if you are setting PARNODES to N, N+1 tasks will be started.

If you are using a queuing system or if you give a list of hosts where TURBOMOLE jobs shall run on (see below), make sure that the number of supplied nodes match $PARNODES -- e.g. if you are using 4 CPUs via a queuing system, make sure that $PARNODES is set to 4.

On IBM systems the total number of tasks in the LoadLeveller script must be set to $PARNODES + 1 except for ricc2.