Browse AWS S3 locations
The contents of AWS S3 buckets you have access to can be browsed by going to:
Element info (
) | Browse S3 locations (
).
If the message "No active S3 locations found." is visible, it means that either no AWS Connections or AWS S3 public buckets have been configured, or that there is no access for the user logged in. Configuring AWS access is described in AWS Connections and AWS S3 public buckets.
Figure 4.3: Browse S3 buckets accessible via the CLC Server. Data can be uploaded, downloaded, deleted and new folders created using functionality in this tab.
When an item in an S3 bucket is selected, URLs to that item are shown below the file listing area and relevant options are enabled (figure 4.2).
The URLs can be copied using the Copy to clipboard button. They can be useful when copying data to import/export directories (see Import/export directories), when sharing the location of files with others, and when specifying inputs and locations to save to using the CLC Server Command Line Tools.
The actions available are:
- Delete files or folders If a folder has been selected, clicking on the Delete... button opens a dialog offering the option to delete the folder and its contents, or to specify individual files from that folder to be deleted. You are asked for confirmation of your intention to proceed with deletion.
- Add a new folder Click on the New Folder... button to create a new folder within the folder selected in the browsing area.
- Download files or folders Data can be downloaded to a CLC File System Location or to an import/export directory.
If a folder has been selected, clicking on the Download... button opens a dialog with the options Download folder..., to download the folder and its contents, and Select files..., to select individual files in that folder for download (figure 4.3).
When selecting individual files, the files in the selected folder are listed, along with their sizes, and the total download size of the selected files (figure 4.4). After download, a list of the downloaded files is presented under the Results area at the bottom of that dialog.
Download starts when the location to download to has been selected.
CLC data downloaded to a CLC location, can be opened using a CLC Workbench connected to the CLC Server and used in downstream analyses.
Figure 4.4: If a folder is selected when the Download... button is clicked, you get the option to download the folder (and its contents) or to select files from within the folder to download.
Figure 4.5: When downloading files from S3, a list of the selected files and the total size of the download is displayed. You will be prompted for a specific location to download to.Note: AWS charges for downloading data from AWS S3.
- Upload data
Data from a CLC File System Location, from an import/export directory or from AWS S3 can be uploaded to the selected folder:
There are two ways to indicate the data to upload:
- Using a graphical file chooser Click on the Add files... button to select files and folders using a graphical file browser. The first step will be to specify the source of the data to upload (figure4.5).
Figure 4.6: Choose the source of data to upload to AWS S3.A graphical file browser will open allowing selection of the files and/or folders to upload.
- Providing a URL URLs for data in a CLC Server File System Location are shown when browsing CLC data in the web client under Element info (
) | Info () and can also be obtained by using the Copy function in the Navigation Area of a Workbench.
URLs for data in an import/export area are shown when browsing import/export directories under Element info (
) | Browse server import/export directories ().
URLs for data in AWS S3 buckets are available when browsing using the web client, as described on this page.
- Using a graphical file chooser Click on the Add files... button to select files and folders using a graphical file browser. The first step will be to specify the source of the data to upload (figure4.5).
The actions that can be taken in practice depend on the permissions for accessing the selected AWS S3 bucket and settings in the CLC Server for the user logged in.
Note: If you are using a CLC Genomics Cloud setup on AWS, non-admin users may see one or more entries called CGC system bucket. These cannot be browsed. They are used for system files, and are not intended for storing sample data or results. When logged in as an admin user, these buckets are listed with their full names, which start with cgc-system. They can be browsed but there is generally no need to do so.