Monitoring and reviewing CLC cloud jobs

CLC jobs run on the cloud are monitored using CloudWatch. Information about jobs can be found via CLC software as well as the AWS Console.

Key files for investigating cloud jobs are:

workflow-result.json This file contains a list of the outputs from an analysis and accessing it via a CLC Workbench or CLC Server.
The analysis log This is the usual source of information about analyses, where the focus is on the analysis steps run. This log provides information useful when investigating problems with the analysis itself. The filename include the name of the analysis followed by "log", for example, "Prepare Raw Data log" (figure 7.2).
Execution Log This file contains the log stream information from Cloud Watch for the job. This file can be opened from the list provided in the Processes tab (figure 7.1), or by double-clicking on the workflow-result.json file in the Remote Files tab (figure 7.2). For very long logs, the first 10,000 entries from the log stream are included, as well as a few entries from the end of the log stream. If the full log is needed, it can be accessed directly in AWS CloudWatch.
The Execution Log can also be retrieved, as long as the information is still available on AWS.
The Execution Log is primarily for troubleshooting problems with a job on the cloud.

In this chapter, we describe monitoring jobs using:

Browse the manual