Cloud connections via a CLC Server
Workflows can be submitted to run on the CLC Genomics Cloud Engine(GCE) via the CLC Genomics Server. This can be particularly useful when using data stored on the CLC Server, as data will be submitted from the server location to S3 directly, avoiding downloading it to your local system.
Note:
- There are no user permissions on jobs in GCE. This means that GCE users will be able to find each other's jobs, for example by using the Cloud Job Search functionality in their Workbench.
- Submitting jobs to GCE can be restricted to specific groups by configuring permissions for each GCE preset (Configuring GCE presets).
To support submission of workflows to GCE, an administrator must install the Cloud Server Plugin and configure various settings via the CLC Genomics Server web administrative interface, as described in Configuring the Cloud Server Plugin.
Launching jobs to run on a CLC Genomics Cloud Engine via a CLC Server
After the CLC Server has been set up to connect to GCE (Configuring the Cloud Server Plugin), the option "CLC Genomics Cloud Engine (via CLC Server)" becomes available to use in the workflow launch wizards in any CLC Workbench with the Cloud Plugin installed that are connected to the CLC Server.
No settings are required for the Workbench plugin. Submission will use the settings configured in the CLC Server.
When the "CLC Genomics Cloud Engine (via CLC Server)" option is selected, you then select a preset to use from the drop-down menu below it. Hover the mouse cursor over the preset name to see information about the machine specifications and result handling configured for that preset.
When submitting jobs to GCE this way, the user logged into the CLC Server is recorded as submitting the job in the CLC Server audit log. The credentials used when running the job on GCE are those configured in the CLC Server for accessing GCE. The user information in the history of data elements generated on GCE reflects this latter point.
Importing data from Illumina Basespace via a CLC Server
When launching analyses to run on a CLC Server or to run on GCE via the CLC Server, you can select data from Illumina Basespace as input. For this, you need an Illumina BaseSpace account, but no configuration of the CLC Server or the Cloud Server Plugin is necessary.
Importing data from and exporting data to Amazon S3 via a CLC Server
When launching analyses to run on a CLC Server or to run on GCE via the CLC Server, you can select data from an active Amazon S3 location that has been configured in the CLC Server. Documentation about configuring Amazon S3 locations is provided in the CLC Server manual, available from https://resources.qiagenbioinformatics.com/manuals/clcserver/current/admin/User_Manual.pdf
Finding jobs and results submitted via the CLC Server using Cloud Job Search
The Cloud Job Search tool in a CLC Workbench with the Cloud Plugin installed can be used to find jobs and download results from analyses submitted to CLC Genomics Cloud Engine via a CLC Server. To find these jobs, the AWS locations, and the GCE settings specified in the CLC Workbench cloud configuration dialog, should either be empty, in which case the details of the configuration settings for the Cloud Server Plugin will be used, or must match those specified in the Cloud Server Plugin configuration in the CLC Server.
The "Only from logged-in user" option in the Cloud Job Search tool will find jobs submitted by both the user authenticated through the cloud configuration dialog and the user logged into the CLC Server. This is most noticeable if these user names differ.
The Cloud Job Search tool is described further in Cloud Job Search.
Subsections