Using a CLC Server to monitor and review cloud jobs
Information about queued and running jobs submitted via the CLC Server is available under in the Queue tab, under the Management tab of the CLC Server web administrative interface. Information about queued, running and completed processes submitted via the CLC Server is available from the Audit log, also under the Management tab of the web administrative interface. Cloud jobs have a have a CGC link in the Status column (figure 7.3).
Figure 7.3: An audit log entry for a completed job. The CGC links in the Status column indicate this was a job sent to the cloud for execution.
When a cloud job is in progress, clicking on the CGC link beside a job in the Queue or in the Status column of the audit log opens a dialog. Before the job starts running, the dialog gives some basic information and a link to the results folder (figure 7.4). While a job is being executed, the AWS log viewer will be opened, allowing progress of the job to be monitored via the CloudWatch log. Clicking on the Reload log button reloads the log and scrolls to latest entry. Search functionality is available at the bottom left (figure 7.5).
Figure 7.4: Clicking on the CGC link for a queued job opens a dialog with basic information and a link to the results folder.
Figure 7.5: Clicking on the CGC link for a running job opens the AWS log viewer. Contents can be searched by typing terms into the field at the bottom left. Clicking on the Reload log button reloads the log and scrolls to the latest entry.
When a job is complete, the word Done will be present in an audit log entry for that job. Clicking on a link in red text in that row opens a window containing detailed information about the job. Drill down through the Server Command Output to get to buttons for downloading the AWS log, i.e. the Cloud Watch log stream for that job, as well as buttons for downloading results or downloading a copy of the workflow that was run.
Figure 7.6: In the Command done window, buttons are provided under Server Command Output that can be used to view the AWS log, i.e. the CloudWatch log stream, as well as for downloading results or the workflow run. Links to input data and the results folder are also available.
Clicking on the CGC link in the Status column of completed jobs gives direct access to the Cloud Workflow Result section of the audit log entry, as does navigating to a workflow-result.json file in the Browse S3 locations area under the Element info tab of the CLC Server web interface and clicking on the View workflow result... button (figure 7.7).
Figure 7.7: Buttons to download workflow results and the AWS log, i.e. the CloudWatch log stream, can be revealed by navigating to a workflow-result.json file in the Browse S3 locations area and clicking on the "View workflow result..." button.
By default, the Audit Log and Queue tabs in the CLC Server web interface are only available to administrative users. Access can be granted to users of other groups, as described at