Workflow distribution options
Workflow distribution options are found under:
Configuration (
) | Job processing (
) | Workflow settings
The three available options, described further below, reflect the three organizational levels of workflows:
- Tasks The individual tasks that comprise the workflow
- The whole workflow All tasks in the workflow
- Blocks Sections of a workflow. For workflows without control flow elements, the entire workflow is a single block. For workflows with control flow elements:
- Tasks downstream of an Iteration element and above a Collect and Distribute element form a block.
- Tasks downstream of an Iteration element with no subsequent Collect and Distribute element form a block.
- Linked workflow steps outside an iteration block form a block
See figure 6.11.
Further details about control flow elements are provided in the Workflow chapter of the CLC Genomics Workbench manual available in html or pdf format from
https://digitalinsights.qiagen.com/technical-support/manuals/
Figure 6.11: This workflow has one iteration block, shaded in turquoise, and one block after the iteration block, shaded in purple. It also has an optional block above the iteration block, outlined in purple. If a user chooses to import files as the initial action taken when launching this workflow, the import would be executed as an initial block. If they choose files already in a CLC File Location, the first block is the iteration block.
The workflow distribution options (figure 6.12) are:
- Submit individual tasks to any available node Each task of a workflow is scheduled separately for execution. For example, a workflow with 10 tasks would result in 10 jobs being submitted. Each of those jobs can be sent to any available node with adequate resources when that step is ready to be run.
- Submit all tasks to a single node A single job submission is made for a workflow, regardless of how many tasks that workflow consists of. All tasks of that workflow are run on the same node. This option was previously called "Single entity".
- Submit tasks in each workflow block to a single node Each iteration of a block of a workflow, and each additional block outside iteration blocks, is scheduled separately for execution. Each job can be sent to any available node with adequate resources when that block is ready to be run. For workflows consisting of just one block, (no control flow elements), this option behaves just like the Submit all tasks to a single node option.
Further details about considerations when choosing a workflow distribution option are provided in Choosing a workflow distribution option.
Figure 6.12: Workflow distribution options determine how workflow tasks are distributed on servers with execution nodes.