Advanced jobs

Nested batch scripts

If you have lots of subjobs, it is recommended that you create a master job controlling various subjobs. E.g. you can split up a big fasta file containing lots of sequences before submitting subjobs, each handling one sequence. At the end you can clean up and/or collect the results. The -sync flag tells qsub to wait until all 100 subjobs are finished before continuing in the script.

 #$ -S /bin/sh
#$ -q nonofficehours.q
#$ -P test
#$ -cwd
echo "preparing the directories etc."
# do the preparation
qsub -sync y -t 1:100
echo "now the jobs are finished - cleaning up"
# do the cleanup

The job will be submitted as follows:


Job chains

An alternative to the nested jobs scripts is a job chain. To use this feature you must give your jobs a common name (e.g. Phase1). The following job in the chain will wait until the referring job is finished. Please note, that you will not know if the previous job in the chain failed. For this you must use nested jobs as described above.

So add to script 1:

 #$ -N Phase1

And add to script 2:

 #$ -N Phase2
#$ -hold_jid Phase1