Multi-threaded programs


Multi-threaded programs can use 1 to the maximum cores of 1 node. To avoid overloading the node, we have to tell the Grid Engine master that the program uses more than 1 core of this nodes resources.

Hardcoded cores request

This can be done in a static way:

 #$ -pe hostcores 6

It means the program in the script uses exactly 6 cores of the node. The job will only start running if the 6 cores are available, otherwise it will wait until a node with 6 cores is free.

Dynamic cores request

A better solution (if your program supports it) 

 #$ -pe hostcores 6-12

This means the program in the script needs minimum 6 cores and maximum 12 cores to run. The job is queued and will be sent to a host if the minimum requirement (6 cores) is available. Based on the cluster load the resources can be more than 6 cores. In this case the Grid Engine sets a special environment variable $NSLOTS for the job's execution environment. The value in $NSLOTS is the actual reserved amount of cores (e.g. 10 cores) for this job.

Below is a practial example on how to use this:

 #$ -S /bin/sh
 #$ -q nonofficehours.q
 #$ -P test
 #$ -cwd
 #$ -pe hostcores 6-12
 echo "got $NSLOTS cores on this node"
 module load ncbi-blast
 blastall -p blastp -a $NSLOTS -i n179.fa -d nr -e 0.01 -v 500 -b 500 -o n179.$NSLOTS.out

So this script lets the Grid Engine assign the free cores on the node and tells the blastall program to start with the given amount of cores.