Using a CLC Server to monitor and review cloud jobs

Information about queued and running jobs submitted via the CLC Server can be seen in the Queue tab, under the Management tab of the CLC Server web administrative interface. Detailed job information for completed processes is available from the Audit log, also under the Management tab of the web administrative interface. Cloud jobs have a C in the Status column (figure 7.3).

Image clcserver-auditlog-cloudjob
Figure 7.3: Jobs sent to the cloud via a CLC Server are recorded in the audit log. A C in the status column indicates a job sent to the cloud. Clicking on the links (red text) opens up information about the job.

In the audit log, click on a link in the row for a job that contains the word "Done" to open a window containing detailed information about the job. Drill down through the Server Command Output to get to buttons for downloading the AWS log, i.e. the Cloud Watch log stream for that job, as well as buttons for downloading results or downloading a copy of the workflow that was run.

Image clcserver-auditlogdeatails-cloudjob
Figure 7.4: In the Command done window, buttons are provided under Server Command Output that can be used to view the AWS log, i.e. the CloudWatch log stream, as well as for downloading results or the workflow run.

The same information can be obtained by navigating to a workflow-result.json file in the Browse S3 locations area under the Element info tab of the CLC Server web interface and clicking on the View workflow result... button (figure 7.5).

Image server-workflowlog-from-s3browser
Figure 7.5: Buttons to download workflow results and the AWS log, i.e. the CloudWatch log stream, can be revealed by navigating to a workflow-result.json file in the Browse S3 locations area and clicking on the "View workflow result..." button.

By default, the Audit Log and Queue tabs in the CLC Server web interface are only available to administrative users. Access can be granted to users of other groups, as described at

https://resources.qiagenbioinformatics.com/manuals/clcserver/current/admin/index.php?manual=Controlling_access_server_server_tasks_external_data.html.