AWS Connections

AWS connections are used when:

Working with data on S3 via the Workbench is of particular relevance when submitting jobs to run on a CLC Genomics Cloud setup making use of functionality provided by the CLC Cloud Module.

When launching workflows to run locally using on-the-fly import and selecting files from AWS S3, the files selected are first downloaded to a temporary folder and are subsequently imported.

Configuring AWS credentials

To configure AWS accounts for data import and export, go to:

        Connections | AWS Connections (Image cloud_access_16_n_p)

After a connection has been configured, it will look like that in figure 6.3, where AWS connections are listed, along with information about their status. These can be edited or removed, if desired. The status is indicated using colors. Green indicates the connection is valid and is ready for use.

Connections to a CLC Genomics Cloud are indicated in the CGC column. To submit analyses the CLC Genomics Cloud, the CLC Cloud Module must be installed and a license for that module available.

Image aws_connection_dialog
Figure 6.3: The configuration dialog for AWS connections. A valid connection has been configured and S3 locations will be available via exporters and relevant importers in the Workbench.

Click on the Add AWS Connection button to configure an AWS connection. The following information should be entered in the configuration dialog (figure 6.4):

The dialog continually validates the settings that have been entered. When the settings are valid, the Status box will contain the text "Valid" and a green icon will be shown. Click on OK to save the settings.

Image aws_connection_configure
Figure 6.4: Adding an AWS account configuration dialog

Importing data from AWS S3

Configured AWS S3 locations will be available in the workflow wizard when using on-the-fly import in workflows, and in relevant import tool wizards (figure 6.5).

Image import_from_aws_location
Figure 6.5: Selecting an AWS connection so that files stored in an S3 location accessible from that account can be selected for import by the Illumina importer of the CLC Genomics Workbench.

Exporting data to AWS S3

To export data to an AWS S3 location, launch the exporter, and when promptd for an export location, select the relevant option from the drop-down menu (figure 6.6).

Image export_to_aws_location
Figure 6.6: After an AWS connection is selected when exporting, you can select the S3 bucket and location within that bucket to export to.