AWS Connections
AWS connections are used when:
- Accessing AWS S3 locations, to import data from or export data to.
- Submitting analyses to a CLC Genomics Cloud setup, if available on that AWS account.
Working with data on S3 via the Workbench is of particular relevance when submitting jobs to run on a CLC Genomics Cloud setup making use of functionality provided by the CLC Cloud Module.
When launching workflows to run locally using on-the-fly import and selecting files from AWS S3, the files selected are first downloaded to a temporary folder and are subsequently imported.
Configuring AWS credentials
To configure AWS accounts for data import and export, go to:
Connections | AWS Connections (
)
After a connection has been configured, it will look like that in figure 6.3, where AWS connections are listed, along with information about their status. These can be edited or removed, if desired. The status is indicated using colors. Green indicates the connection is valid and is ready for use.
Connections to a CLC Genomics Cloud are indicated in the CGC column. To submit analyses the CLC Genomics Cloud, the CLC Cloud Module must be installed and a license for that module available.
Figure 6.3: The configuration dialog for AWS connections. A valid connection has been configured and S3 locations will be available via exporters and relevant importers in the Workbench.
Click on the Add AWS Connection button to configure an AWS connection. The following information should be entered in the configuration dialog (figure 6.4):
- Connection name: A short name of your choice, identifying the AWS account. This name will be shown as the name of the data location when importing data to or exporting data from Amazon S3.
- Description: A description of the AWS account (optional).
- AWS access key ID: The access key ID for programmatic access for your AWS IAM user.
- AWS secret access key: The secret access key for programmatic access for your AWS IAM user.
- AWS region: An AWS region. Select from the drop-down list.
- AWS partition: The AWS partition for your account.
The dialog continually validates the settings that have been entered. When the settings are valid, the Status box will contain the text "Valid" and a green icon will be shown. Click on OK to save the settings.
Figure 6.4: Adding an AWS account configuration dialog
Importing data from AWS S3
Configured AWS S3 locations will be available in the workflow wizard when using on-the-fly import in workflows, and in relevant import tool wizards (figure 6.5).
Figure 6.5: Selecting an AWS connection so that files stored in an S3 location accessible from that account can be selected for import by the Illumina importer of the CLC Genomics Workbench.
Exporting data to AWS S3
To export data to an AWS S3 location, launch the exporter, and when promptd for an export location, select the relevant option from the drop-down menu (figure 6.6).
Figure 6.6: After an AWS connection is selected when exporting, you can select the S3 bucket and location within that bucket to export to.