CRC New Environment 2009

This New Environment Quickstart

The CRC New Environment consists of a major software stack upgrade to provide greater capability and performance than available via incremental changes to the old environment. Major upgrades include:

  • Red Hat 4 -> Red Hat 5
  • SGE 5.3 -> SGE 6.2
  • Default AFS cell ->

All new equipment and software applications will be installed only for the CRC New Environment. Existing equipment and software applications will be migrated to the new environment.

To log into the New Environment, use

IMPORTANT: The CRC New Environment 2009 uses a new AFS cell ( run by the CRC staff -
It also uses a new Kerberos 5 Authentication service which is separate from the
AFS environment run by the Notre Dame Office of Information Technologies Group.
Due to that there is a separate password that is different than the 
one used in the cell. We recommend that users synchronize their passwords
automatically by resetting their password on the Notre Dame password page.

FAQ's that users have had

1) Moving files from your AFS cells: to
To utilize your space in the AFS cell you will need to move/manage your files.  
There are mutiple ways to move your files such as the following three examples:
* Use scp from opteron to crcfe02
* Mount both AFS cells on your workstation and perform direct movement
For more information on the AFS cell please reference CRC AFS Cell

2) The Beta test front-end machine is called with aliases to and (you may use either name)
Users may find that they get an error when logging into front-end
machine - this is due to the fact that the machine was recreated without the
old ssh keys being saved so you may see the following error message:

Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that the RSA host key has just been changed.
The fingerprint for the RSA key sent by the remote host is
Please contact your system administrator.
Add correct host key in ********** to get rid of this message.
Offending key in **********
RSA host key for has changed and you have requested strict checking.
Host key verification failed.

If this is the case simply edit your ~/.ssh/known_hosts file or 
remove the file and ssh again.

3) When logging into there are few modules loaded by default -
Users should load modules in their script files for access to software

Also note that you should load and use the default module unless you have a specific 
need for a different version

For Example: 
"module load matlab"
rather than 
module load matlab/7.7

If you might use matlab/7.7 your scripts will then generate errors as 
version 7.7 is removed at a later date.

4) You'll find your batch scripts which use the wildcard * queue naming (such as ompi-*) no longer work.

This is due to a renaming of the mpi queues - you will need to
modify / submit jobs now using the mpich1 or ompi queue 

For example replace 
#$ -pe ompi64-* 16
#$ -pe mpich1 16
#$ -pe ompi 16

5) which command should I use in the X-win configuration for crcfe02?
It does not work with the same configuration as for opteron
The command filed should read:
instead of /usr/X11R6/bin/xterm

Update to #5: Due to some hardcoded binaries - the "new" environment now 
has a link created for /usr/X11R6/bin/xterm

{rich}5: ls -l /usr/X11R6/bin/xterm
lrwxrwxrwx 1 root root 14 May  5 16:29 /usr/X11R6/bin/xterm -> /usr/bin/xterm

6) Error in the output of fs listquota

With the upgrade to OpenAFS 1.4.10 on the CRC fileservers partition
sizes of greater than 2 TB are supported (not user volumes which are
still limited to 2 TB).  However the internal workings of the OpenAFS
fs commands are still limited to 32 bits due to a protocol change 
necessary -  so the end result is that you MAY see a wierd negative 
number reported for the partition size used on the fileserver if it's
over 2 TB - for instance{rich}9: fs listquota .
Volume Name                   Quota      Used %Used   Partition
u.jducom                  100000000   3238439    3%       -608%{rich}10: 

The -608% is incorrect - we recommend that this information 
returned for the Partition quota used be ignored.