Workflow queuing options
Workflow queuing options are found under:
Configuration (
) | Job distribution (
)
There are three options for how workflows should be submitted to nodes (figure 6.13). These options reflect the three organizational levels of workflows:
- Tasks The individual tasks that comprise the workflow
- The whole workflow All tasks in the workflow
- Blocks Sections of a workflow. For workflows without control flow elements, the entire workflow is a single block. For workflows with control flow elements:
- Tasks downstream of an Iteration element and above a Collect and Distribute element form a block.
- Tasks downstream of an Iteration element with no subsequent Collect and Distribute element form a block.
- Linked workflow steps outside an iteration block form a block
See figure 6.12.
Further details about control flow elements are provided in the Workflow chapter of the CLC Genomics Workbench manual available in html or pdf format from
https://digitalinsights.qiagen.com/technical-support/manuals/
Figure 6.12: This workflow has one iteration block, shaded in turquoise, and one block after the iteration block, shaded in purple. It also has an optional block above the iteration block, outlined in purple. If a user chooses to import files as the initial action taken when launching this workflow, the import would be executed as an initial block. If they choose files already in a CLC File Location, the first block is the iteration block.
The worklow queuing options are:
- Submit individual tasks to any available node Each task of a workflow is scheduled separately for execution. For example, a workflow with 10 tasks would result in 10 jobs being submitted. Each of those jobs can be sent to any available node with adequate resources when that step is ready to be run.
- Submit all tasks to a single node A single job submission is made for a workflow, regardless of how many tasks that workflow consists of. All tasks of that workflow are run on the same node. This option was previously called "Single entity".
- Submit tasks in each workflow block to a single node Each iteration of a block of a workflow, and each additional block outside iteration blocks, is scheduled separately for execution. Each job can be sent to any available node with adequate resources when that block is ready to be run. For workflows consisting of just one block, (no control flow elements), this option behaves just like the Submit all tasks to a single node option.
Further information about workflow queuing options is provided in Choosing a workflow queuing option.
Figure 6.13: Workflow queuing options determine how workflow jobs are queued on servers with execution nodes.
Subsections