• Product manuals

Browse the manual

• Introduction to the CLC Genomics Cloud
  • Overview of CLC Genomics Cloud
  • CLC Genomics Cloud requirements
  • Contact information
  • Installing the CLC Cloud Module
    • Licensing modules
    • Uninstalling modules
  • Installing the Cloud Server Plugin
• Setting up CLC Genomics Cloud resources on AWS
  • Setting up AWS resources
  • Adding more AWS Batch queues for CLC jobs
  • Adding more AWS IAM users
  • Deleting or redeploying CLC Genomics Cloud infrastructure on AWS
• Using a CLC Workbench to work on the cloud
  • Prerequisites for working on the cloud using a Workbench
  • Configuring an AWS Connection in the Workbench
  • Submitting analyses to AWS from a Workbench
  • Working with AWS S3 using the Remote Files tab
  • Accessing results from the Processes tab
• Using a CLC Server to work on the cloud
  • Benefits of submitting jobs via a CLC Server
  • AWS Connections in the CLC Server
  • Configuring cloud presets
  • Browse AWS S3 locations
  • Submitting workflows to the cloud via a CLC Server using a CLC Workbench
  • Submitting workflows to the cloud using the CLC Server Command Line Tools
• Integrating third party tools into CLC software
  • Installing external applications in a CLC Workbench
• General information about inputs and results
  • General information about input data for cloud analyses
    • Reference data for analyses on the cloud
    • Using locally held data for analyses on AWS S3
  • General information about outputs from cloud analyses
• Monitoring and reviewing CLC cloud jobs
  • Using a CLC Workbench to monitor and review cloud jobs
  • Using a CLC Server to monitor and review cloud jobs
  • Using the AWS Console to monitor and review cloud jobs
• Appendix
  • Standard AWS Batch queues for CLC Genomics Cloud
  • AWS Batch queues, compute environments, AMIs and images
  • System file retention policies
  • Making custom plugins available for cloud analyses

Using locally held data for analyses on AWS S3

Two considerations when using data stored locally for analysis on AWS S3 are:

1. Transfer of data from a non-cloud location, such as from CLC Locations, from local directories, or from CLC Server import/export directories, takes time, and when submitting jobs from the CLC Workbench, any necessary data transfer must be complete before the software can be closed.
2. Data uploaded to AWS S3 as part of job submission is not saved to AWS S3 for subsequent use. Later analyses using the same input data would involve another transfer of that data to the cloud.

Cases where locally held data must be used:

• Launching a tool from the CLC Workbench Toolbox to run on a CLC Genomics Cloud^6.2. This tool will be wrapped in a workflow for you, before the job is sent to the cloud. When launching analyses this way, you can only select data held in CLC Locations. Thus, the first step when a tool has been launched will always be to upload the input data to AWS S3. Launching tools can be useful when running small tests, but for analysis of large data sets, or where data needs to be imported, we recommend creating a workflow containing the tool of interest and launching that instead.
• When specifying reference data elements in certain workflow designs, as described in Reference data for analyses on the cloud
• Running analyses that require a CLC Metadata Table with data associated to it using a workflow that does not include an Iterate element. An example would be a workflow with the Differential Expression tool at the top of the workflow.

Workflows containing Differential Expression can be run using data already on AWS S3 when that workflow contains an Iterate element upstream. An example illustrating this is the RNA-Seq and Differential Gene Expression Analysis template workflow, delivered with the CLC Genomics Workbench, described at https://resources.qiagenbioinformatics.com/manuals/clcgenomicsworkbench/current/index.php?manual=RNA_Seq_Differential_Gene_Expression_Analysis_workflow.html . If the batch units for the the RNA-Seq analysis section of that workflow are defined using metadata, that same metadata will be used as input to the Differential Expression step.

---

Footnotes

... Cloud^6.2

Only tools that can be used within workflows can be submitted to run on a CLC Genomics Cloud. Tools that are not workflow-enabled cannot be run on the cloud.

---