Workflow queuing options
In the Workflow queuing options section under the Job distribution tab (figure 6.13), there are three options for how workflows should be submitted to nodes. These options reflect the three organizational levels of workflows:
- Tasks The individual tasks that comprise the workflow
- The whole workflow All tasks in the workflow
- Blocks Sections of a workflow. For workflows without control flow elements, the entire workflow is a single block. For workflows with control flow elements:
- Tasks downstream of an Iteration element and above a Collect and Distribute element form a block.
- Tasks downstream of an Iteration element with no subsequent Collect and Distribute element form a block.
- Linked workflow steps outside an iteration block form a block
See figure 6.12.
Further details about control flow elements are provided in the Workflow chapter of the CLC Genomics Workbench manual available in html or pdf format from
https://www.qiagenbioinformatics.com/support/manuals/
Figure 6.12: This workflow has one iteration block, shaded in turquoise, and one block after the iteration block, shaded in purple. It also has an optional block above the iteration block, outlined in purple. If a user chooses to import files as the intial action taken when launching this workflow, the import would be executed as an initial block. If they choose files already in a CLC File Location, the first block is the iteration block.
The worklow queuing options are:
- Submit individual tasks to any available node A CLC Grid Worker is launched for each task of a workflow. For example, a workflow with 10 tasks would result in 10 CLC Grid Workers being launched. Each of these jobs can be sent to any available node with adequate resources when that step is ready to be run. This option was previously called "Classic".
- Submit all tasks to a single node A single CLC Grid Worker is launched for the workflow run. All tasks of the workflow are executed on the same node. This option was previously called "Single entity".
- Submit tasks in each workflow block to a single node A CLC Grid Worker is launched for each iteration of a block and a CLC Grid Worker is launched for each additional block outside the iteration block(s). Each job can be sent to any available node with adequate resources when that block is ready to be run. For workflows consisting of just one block, (no control flow elements), this option behaves just like the Submit all tasks to a single node option.
Figure 6.13: Workflow queuing options determine how workflow jobs are queued on servers with execution nodes.
Subsections