Multi-job Processing

Jobs will be executed on job nodes that have enough available resources to execute the given job. Multiple jobs can run at the same time on a single job node or single server If you want to disable this feature, you can do so by going to the administrative interface of the master server:

        Admin (Image preferences) | Main Configuration (Image preferences) | Job Queuing Options

Then set "Multi-job Processing" to "Disable" (see figure 3.8) and click on the button labeled "Save". This will ensure that only one task will be executed at a given time on a job node or single server.

Image manual-enable-disable-2
Figure 3.8: Select "Disable" to force that only a single job is executed at a time on a job node or single server.

Not all jobs can run at the same time, e.g. the "Map Reads to Reference" job usually requires too many resources for it to make sense to be run concurrently with other jobs, thus it requires a single job node to itself. Likewise, some seemingly simple tasks with high amounts of disk access, such as "Convert from tracks" that practically only reads data and writes it to a different format, will not be run concurrently with other jobs of this type, as its disk access is so intense that nothing will be gained by running them concurrently. See Appendix Non-exclusive Algorithms for a list of algorithms that can be executed concurrently on the same job node or single server.

As job nodes can be partially utilized, situations may occur where the job that is the next in line to be executed, cannot be run because it requires more resources than available. In that case, other jobs in the queue will be allowed to run if the available resources allow it. However, in order to avoid that the job in front of the queue is never run, the server keeps track of how many times a job has been overtaken by smaller jobs. When the job has been overtaken too many times, a job node is reserved for it, meaning that smaller jobs are not allowed to run before the job has been run.

The number of times a job can be overtaken, before the system reserves resources for it, is controlled by a Fairness Factor. The default value of this is 10, meaning that 10 jobs are allowed to overtake a larger job, before the larger job is forced to run. The value can be changed:

        Admin (Image preferences) | Main Configuration (Image preferences) | Job Queuing Options

Then change the Fairness Factor to the desired value.