Import/export directories
Import/export directories are designated areas that the CLC Server can write to or to read from. Primary uses of import/export directories are storing data that will be imported to a CLC Server File System Location or storing data exported from the CLC Server. They are also specified for directories where BLAST databases that should be accessible from CLC software are stored, and for the directory where temporary files will be written when using functionality requiring direct data transfer from client systems.
To configure an import/export directory, go to:
Configuration (
) | External data (
) | Import/export directories
Click on the Add new import/export directory button and enter the relevant path (figure 8.2). The specified folder and its subfolders will then be available to client software logged into the CLC Server when relevant activities are carried out.
Requirements for import/export directories
- Folders configured as import/export directories must be readable and writable by the user that runs the CLC Server process. A user logged into the CLC Server can access files stored in, or export files to, an import/export directory or its subdirectories via CLC client software, however, it is the user running the server process that actually interacts with the file system.
- A directory already configured as a CLC Server File System Location, or a subdirectory of that location, should not be used as an import/export directory.
- On setups with execution nodes, an area configured as import/export directory must be a shared directory, accessible from the nodes and from the master server.
Figure 8.2: Areas external to the CLC Server that it can read data from and write data to, are configured as import/export directories in the External data area.
Access to import/export directories
By default, all users have access via CLC client software to areas configured as import/export directories. Access can be limited by setting permissions under:
Configuration () | Global permissions (
) | Import/export directories
These permissions affect which groups can import data from, or export data to, an import/export directory, and, for BLAST database locations, which groups can list and use BLAST databases in that directory.
Note that permissions set on an import/export directory used for direct data transfer from client systems do not affect the use of this directory to store temporary files created for the purposes of direct data transfer. Where relevant, permissions can be set for the specific functionality. For example, by default, only members of the admin group can install workflows on the server, irrespective of permissions set on the import/export directory that has been assigned for direct data transfer use. Extending permissions for installing workflows is done under the Global permissions tab.
Browsing import/export areas
The contents of import/export directories can be browsed in the web client under:
Element info (
) | Browse server import/export directories ()
When a folder or file is selected, information about it is displayed below the file browser (figure 8.3).
URLs for files in import/export locations can be copied using the Copy to clipboard button. CLC URLs can be useful when copying data to AWS S3 locations (see Browsing AWS S3 locations), when sharing the location of files with others that have access to the same system, and when specifying files to be imported, or locations to export to, using the CLC Server Command Line Tools. When a Web URL is entered into the address bar of a web browser, the web client opens with the relevant item under the "Browse server import/export directories" tab.
Some data management tasks can be carried out from in this area. Folders or files can be deleted, new folders can be created, and files can be copied from AWS S3 locations or CLC File Server Locations into a selected folder by pasting a file URL into the text field near the bottom, replacing the Paste URL here... text.
URLs for data in AWS S3 buckets are shown when browsing under
Element info () | Browse S3 locations ().
Data from a CLC File Server Locations is normally saved to an import/export directory when using export functionality.
URLs that can be used to copy data directly from a CLC Server File System Location are shown when browsing CLC data in the web client under Element info (
) | Info () and can also be obtained by using the Copy function in the Navigation Area of a Workbench.
Figure 8.3: Browsing a CLC Server import/export directory in the web client.