D16copt034-d16copt037

Cluster Details

Cluster PI

  • Prof. Shahriar Mobashery
Description Nodes Cores/Node Total Cores Memory/Node
HP Proliant SL165z G7 servers 4 32 128 24GB

Processor Details

  • Dual 16-core 2.1 GHz AMD Opteron (Interlagos) processors

Network Fabric

  • 1Gb Ethernet

Grid Engine Details

Hostnames

  • d16copt034 - d16copt037

Target Hostgroup

  • @mobashery

Example Submission Script

#!/bin/csh
 
#$ -M netid@nd.edu          # Email address for job notification
#$ -m abe                   # Send mail when job begins, ends and aborts
#$ -pe mpi-32 128           # Specify parallel environment and legal core size
#$ -q *@@mobashery         # Specify queue
#$ -N jobname               # Specify job name
 
# Run application
mpiexec -n 128 ./app.exe

Code Compilation and Performance

Ethernet

CRC provides the MPICH2 and OpenMPI compilers for compiling your MPI codes to run over the Ethernet network. These compilers can be selected on CRC systems as follows:

module load mpich2
module load openmpi

Resource Idleness and Sharing

In the interest of maximizing available computing resources for CRC users, the CRC implements an overflow sharing policy for idle nodes on faculty-partnership clusters. If there are users waiting in the general queue to run jobs, and faculty cluster nodes are idle, CRC staff will periodically allow the waiting users to migrate their jobs to your idle cluster nodes to alleviate backlog and improve system utilization.

Note:

  • these migrated jobs can be cancelled at the request of the cluster PI via direct correspondence with CRC Support
  • the cancellation procedure allows migrated jobs a maximum of four (4) hours to complete before they are killed by CRC staff

Cluster Benchmarks

STREAM

32 Threads
The total memory requirement is      22379 MB
Function     Rate (MB/s)  Avg time   Min time  Max time 
Copy:      30814.1618      0.5154      0.5077      0.5298
Scale:     18702.7567      0.8476      0.8365      0.8828
Add:       20605.1292      1.1446      1.1389      1.1626
Triad:     20569.1133      1.1482      1.1409      1.1520
1 Thread
The total memory requirement is      22379 MB
Function     Rate (MB/s)  Avg time   Min time  Max time 
Copy:       7164.7651      2.2130      2.1835      2.2432
Scale:      4219.8252      3.7392      3.7074      3.7667 
Add:        4360.5461      5.4030      5.3816      5.4350
Triad:      4334.8002      5.4381      5.4136      5.4647