Coarray Fortran

Loading The Intel Cluster Studio Evaluation Module

To load the Intel Cluster Studio evaluation module, perform the following from the CRC command line:

> module use /opt/crc/sandbox/Test_Modules/
> module load intel/ics-eval

Compiling Coarray Fortran (CAF) Source

Compiling For A Single (Shared-Memory) Node

To compile a CAF code for a single (shared-memory) node and an arbitrary number of images (to be defined at runtime):

> ifort -coarray=shared -o ./hello_image hello_image.f90

Compiling For Multiple (Distributed) Nodes

To compile a CAF code to run over multiple distributed nodes and an arbitrary number of images (to be defined at runtime):

> ifort -coarray=distributed -o ./hello_image hello_image.f90

Defining Number of Images at Compile-Time

To fix the number of CAF images at compile-time (e.g. 8):

> ifort -coarray=distributed -coarray-num-images=8 -o ./hello_image hello_image.f90

Modifying Number of Images at Run-Time

To set the number of images to be used at runtime, use the following environment variable:

> setenv FOR_COARRAY_NUM_IMAGES 32 #csh shell
> export FOR_COARRAY_NUM_IMAGES=32 #bash shell

Submitting (Distributed) CAF Jobs To The Grid Engine

Submitting CAF jobs (which has been compiled for a distributed system) to the CRC Grid Engine requires a modified submission script as follows:

CAF Script Template

#!/bin/csh
#$ -M netid@nd.edu 
#$ -m abe
#$ -pe mpi-12 36
#$ -N caf_test
#$ -q long

# Load the Intel Cluster Studio
module use /opt/crc/sandbox/Test_Modules/intel
module load ics-eval

# Generate Host File For Intel MPI
set HOSTFILE=`mktemp /tmp/hostfile.XXXX` || exit 1
cat $PE_HOSTFILE | cut -f1 -d" " >> $HOSTFILE

# Boot the Intel MPI Runtime (totalnum=nodes, ncpus=cores)
mpdboot --totalnum=3 --ncpus=36 -f $HOSTFILE

# Set Number of CAF Images at Runtime
setenv FOR_COARRAY_NUM_IMAGES 36
 
# Execute CAF code
./hello_image

# Exit/Kill Intel MPI Runtime 
mpdallexit