Job Distribution
The CLC Server has the concept of distributing jobs to nodes. This means that you can have a master server with the primary purpose of handling user access, serving data to users and starting jobs, and you have a number of nodes that will execute these jobs.
Three models are available:
- Model I: Master server with dedicated job nodes. In this model, a master server submits CLC jobs directly to machines running the CLC Server for execution. In this setup, a group of machines (from two upwards) have the CLC Server software installed on them. The system administrator assigns one of them as the master node. The master controls the queue and distribution of jobs and compute resources. The other nodes are job nodes, which execute the computational tasks they are assigned. This model is simple to set up and maintain, with no other software required. However, it is not well suited to situations where the compute resources are shared with other systems because there is no mechanism for managing the load on the computer. This setup works best when the execute nodes are machines dedicated to running a CLC Server. Further details about this setup can be found in Setting up execution nodes
- Model II: Master server submitting to grid nodes. In this model, a master server submits tasks to a local third party scheduler. That scheduler controls the resources on a local computer cluster (grid) where the job will be executed. This means that it is the responsibility of the native grid job scheduling system to start the job. When the job is started on one of the grid nodes, a CLC Grid Worker, which is a stand-alone executable including all the algorithms on the server, is started with a set of parameters specified by the user. Further details about this setup can be found in Setting up grid integration.
- Model III: Single Server setup. In this model, the master and execution node functionality is carried out by a single CLC Server instance.
Figure 6.1 shows a schematic overview.
For models I and II, the master server and job nodes, or master server and grid nodes must run on the same type of operating system. It is not possible to have a master server running Linux and a job node running Windows, for example.
Figure 6.1: An overview of the job distribution possibilities.
Subsections
- Model I: Master server with dedicated job nodes
- Model II: Master server submitting to grid nodes
- Model III: Single Server setup