Understanding memory settings

Most work done by the CLC Server is done via its java process. However, some tools use external native binaries for the computational phases. These include those with a de novo assembly or mapping phases and those involving BLAST tools.

Java process

Each grid preset can have its own memory limit for the java process. This can be useful where separate queues are used for low overheads tasks, such as import jobs and trimming jobs, and high overhead tasks, such as de novo assemblies or read mappings.

Unless a memory limit is explicitly configured for a grid worker, auto-detection is used to determine the value to apply. This should be appropriate for most circumstances. The limit is determined as follows:

Order of priority for java process memory settings The first value found according to the list below will be applied:

  1. A memory limit specified in a clcgridworker.vmoptions file. This method is not recommended.
  2. A memory limit specified in a CLC Grid Preset. If values determined by auto-detection are not appropriate, this method can be used to specify a custom value. See Configure grid presets.
  3. Auto-detection, as described above.

External binaries

Some tools make use of external binaries during one or more phases. Memory usage by external binaries is not restricted by limits set for the java process. For this reason, we recommend caution if you plan to submit jobs of these types to nodes that are being used simultaneously for other work.