Technical overview

Figure 6.4 shows an overview of the communication involved in running a job on the grid, using OGE as the example submission system.

Image ogeoverview
Figure 6.4: An overview of grid integration, using OGE as the example submission system.

The steps of this figure are in detail:

  1. From the Workbench the user invokes an algorithm to be run on the grid. This information is sent to the master server running the CLC Server.
  2. The master server writes a file with job parameters to a shared work directory of the grid execution nodes. The job parameters contain identifiers mapping to the job data placed in the CLC server data location. The job parameters file is automatically deleted when it is no longer used by the grid node.
  3. Now the server invokes qsub through the specified DRMAA native library. Then qsub transfers the job request to the grid scheduler. Since the user that runs the CLC Server process has invoked qsub, the grid node will run the job as this CLC-Server user.
  4. The job scheduler will choose a grid node based on the parameters given to qsub and the user that invoked qsub.
  5. The chosen grid node will retrieve CLC Grid Worker executable and the job parameters from the shared file system and start performing the given task.
  6. After completion of the job, the grid node will write the results to the server's data location. After this step the result can be accessed by the Workbench user through the master server.