D16copt001-d16copt033

Cluster Details

Cluster PI

  • Dr. TengFei Luo
Description Nodes Cores/Node Total Cores Memory/Node
HP Proliant SL165z G7 servers 33 32 1056 24GB

Processor Details

  • Dual 16-core 2.1 GHz AMD Opteron (Interlagos) processors

Network Fabric

  • 1Gb Ethernet

Grid Engine Details

Hostnames

  • d16copt001 - d16copt033

Target Hostgroup

  • @tengfeiluo

Example Submission Script

#!/bin/csh
 
#$ -M netid@nd.edu          # Email address for job notification
#$ -m abe                   # Send mail when job begins, ends and aborts
#$ -pe mpi-32 128           # Specify parallel environment and legal core size
#$ -q *@@tengfeiluo         # Specify queue
#$ -N jobname               # Specify job name
 
# Run application
mpiexec -n 128 ./app.exe

Code Compilation and Performance

Ethernet

CRC provides the MPICH2 and OpenMPI compilers for compiling your MPI codes to run over the Ethernet network. These compilers can be selected on CRC systems as follows:

module load mpich2
module load openmpi

Resource Idleness and Sharing

In the interest of maximizing available computing resources for CRC users, the CRC implements an overflow sharing policy for idle nodes on faculty-partnership clusters. If there are users waiting in the general queue to run jobs, and faculty cluster nodes are idle, CRC staff will periodically allow the waiting users to migrate their jobs to your idle cluster nodes to alleviate backlog and improve system utilization.

Note:

  • these migrated jobs can be cancelled at the request of the cluster PI via direct correspondence with CRC Support
  • the cancellation procedure allows migrated jobs a maximum of four (4) hours to complete before they are killed by CRC staff

Cluster Benchmarks

LAMMPS

ESPRESSO

STREAM

32 Threads

Function     Rate (MB/s)  Avg time   Min time  Max time 
Copy:      30752.9080      0.4936      0.4856      0.5123
Scale:     18462.7678      0.8297      0.8088      0.8375 
Add:       19903.2191      1.1343      1.1254      1.1421
Triad:     19935.2814      1.1394      1.1236      1.1469

1 Thread

Function     Rate (MB/s)  Avg time   Min time  Max time
Copy:       6466.5922      2.3119      2.3093      2.3208
Scale:      4075.9961      3.6661      3.6637      3.6810
Add:        4153.7034      5.3945      5.3928      5.4046
Triad:      4139.6059      5.4130      5.4111      5.4232