Simple array jobs

Job Arrays

If you have lots of jobs doing the same task just with different parameters, you can use a so called job array.

So if you want to run the same executables (e.g. blast) with 100 different inputfiles you can create a jobtask (e.g. myblast.sh):

 #$ -S /bin/sh
 #$ -q nonofficehours.q
 #$ -P test
 #$ -cwd
 echo "I am task: $SGE_TASK_ID with jobname: $JOB_NAME running on host: `hostname`"
 module load ncbi-blast
 blastall -p blastp -i $SGE_TASK_ID.fa -d nr -e 0.01 -v 500 -b 500 -o $SGE_TASK_ID.out

The above job-script can be submitted as follows:

 qsub -t 1:100 myblast.sh

If you do not want to use up all resources (e.g. leave computing space for other team members), you can submit these array jobs in e.g. groups of 10 subjobs as follows:

 qsub -t 1:100 -tc 10 myblast.sh

In this case only 10 of the 100 jobs will be submitted first, and when they are finished the next 10 jobs are submitted until all 100 jobs are run.