Launching workflows individually and in batches

When a workflow is launched, a launch wizard appears, allowing you to configure available options.

Workflows stored in a CLC data location, i.e. available from the Navigation Area, can be launched by opening them, and then clicking on the Run Workflow button at the bottom, right hand side of the Workflow Editor.

Installed workflows and template workflows can be launched in the following ways:

Workflow inputs

Inputs can be CLC data elements, selected from the Navigation Area, or files located on any accessible system. This includes CLC Server import/export directories when you are connected to a CLC Server, and files on AWS S3 if you have configured an AWS Connection.

To select files from a non-CLC location, choose the "Select files for on-the-fly import" option in the launch wizard (figure 13.62). When you do this, the files will be imported on the fly, as the first action taken when the workflow runs.

Image on-the-fly-import-combined
Figure 13.62: To select input data stored somewhere other than a CLC location, choose the option "Select files for on-the-fly import" in the launch wizard.

Further details about workflow inputs are provided in Configuring Workflow Input elements.

Workflow outputs

Output and Export elements in workflows specify the analysis results to be saved. In addition to analysis results, a Workflow Result Metadata table can be output. This contains a record of the workflow outputs, as described in Workflow Result Metadata tables.

The history of data elements generated as workflow outputs contains the name and version the workflow that created it. When an installed workflow was used, the workflow build id is also included (see History and Element Info views).

Launching batches of analyses

When multiple inputs are provided, you can choose to run the workflow multiple times, one time for each input, or for defined sets of inputs that should be analyzed together. This is referred to as running in Batch mode, with the inputs to be analyzed together being referred to as batch units. This is described in Running workflows in batch mode.

In addition, using Iterate and Collect and Distribute control flow elements within a workflow allows for part of a workflow being run once per batch unit, while other parts are run on all the data together. This is described in Running part of a workflow multiple times.

Workflow intermediate results

Workflow intermediate results are generated during the workflow execution. These data elements are needed for use by downstream steps but are then deleted when the analysis successfully completes. While they exist, workflow intermediate results are stored in a dedicated subfolder of the folder that is selected to store outputs when the workflow is launched.

Where intermediate results are stored for jobs run on a CLC Server depends on settings on the server.


Subsections