Controlling the number of cores utilized
This section covers basics related to setting limits on core or thread usage via a CLC Server grid preset. Parameter details are specific to the grid scheduler being used. We provide some information below for some schedulers, but please refer to the grid scheduler documentation for full details.
When using "Legacy mode" grid mode, or when running exclusive jobs with the "Resource Aware" grid mode, the default for all jobs is to assume they have access to all cores on the node they are run on. Details about grid modes can be found in Configure grid presets.
When configuring a core or thread limit for exclusive jobs, i.e. those that will require use of a whole node, the relevant parameter(s) and integer value(s) to be used by the grid scheduler are entered directly in the Native specification field. To specify different core limits for different types of tasks, one could set up multiple presets with different values supplied in the Native specification field of each.
Configuration of core requirements is central to supporting concurrent execution of non-exclusive jobs on a grid node. This is done by specifying core or thread requirements in the Shared native specification, making use of the variables COMMAND_THREAD_MIN, COMMAND_THREAD_MAX and optionally the functions take\_lower\_of and take\_higher\_of. Further information about this is provided in Multi-job processing on grid.
Configuration of OGE
1) CPU Core usage when not using parallel environment
In the CLC Server, there is an environmental variable, which when set to 1, specifies that the number of allocated slots should be interpreted as the maximum number of cores a job should be run on. To set this environmental variable, add the following to the Native specification of the grid preset:
-v CLC_USE_OGE_SLOTS_AS_CORES=1
In this case, the maximum number of cores the job should use will be set to the number of slots allocated by OGE for the job.
2) Limiting CPU core usage when using the parallel environment feature
The parallel environment feature can be used to limit the number of cores used by the CLC Server jobs when running on the grid. When the parallel environments feature is used, the number of allocated slots is interpreted as the maximum number of cores to be used by the job. The parallel environment must be setup by the grid administrator in such a way that the number of slots corresponds to the number of cores.
The syntax in the Native specification for using parallel environments is:
-pe <pe-name> <min-cores>-<max-cores>
where pe-name is the name of the parallel environment, and a range of cores is specified with integers, e.g. 1-4.
An example as might be entered into a Native specification when using parallel environments is:
-l cfl=1 -l qname=64bit -pe clc 1-3.
Here, the clc parallel environment is selected and 1 to 3 cores are requested.
Configuration of PBS Pro
With PBS Pro the number of cores to use is specified with a single number. This request can be granted (the process is scheduled) or denied (the process is not scheduled). The number of cores are requested as a resource: -l nodes=1:ppn=X, where X is the number of cores. Please ensure that the number of nodes requested is 1.
An example as might be entered into a Native specification is: -q bit64 -l nodes=1:ppn=2. This would request 2 cores and the job would be put in the bit64 queue.
Configuration of LSF
With LSF the number of cores to use is specified with the -n option. This parameter can accept a single argument or two arguments. A single argument, -n X, is a request for exactly X cores. Two arguments, -n X,Y, (separated by a comma), is a request for between X and Y cores.