Moving Data From CHPC with smbclient

Overview

This document serves as a guide for current CHPC users who wish to transfer data to RIS storageN.

storageN

  • The use of storageN within these documents indicates that any storage platform can be used.

  • Current available storage platforms:

    • storage1

    • storage2

Prerequisites

  1. Meet the assumptions listed for accessing storageN.

  2. A CHPC account with access to the data node.

  3. A user account for RIS storageN and compute1 services.

Transferring data from CHPC to storageN

This guide will provide a demo transfer of a 1 GB file from the CHPC scratch folder to RIS storageN.

The file in question is named file_1.bin and is located in the CHPC scratch folder /scratch/stahan.

../../_images/chpc-file_1.png

The goal is to transfer the file to a folder in the RIS storageN Active directory named chpc_data.

I. Connecting to the CHPC Data Node

  • Open a terminal session and connect to the CHPC Data Node.

ssh USERNAME@dtn01.chpc.wustl.edu

Where USERNAME is your CHPC username. For more information on connecting via the terminal, including required software, please reference the compute quick start page.

II. Connect to storageN using smbclient

a. Create a file to store smbclient credentials in your home directory.

touch ~/.smb_creds

b. Enter the following into the ~/.smb_creds file using your text editor of choice, substituting WUSTL KEY ID and WUSTL KEY PASSWORD with your own WUSTL Key credentials.

username = WUSTL KEY ID
password = WUSTL KEY PASSWORD
domain = ACCOUNTS.AD.WUSTL.EDU

c. Use smbclient to connect to your storageN allocation replacing WUSTL_KEY with the WUSTL Key that the storage allocation belongs to (e.g., a PI).

smbclient -A ~/.smb_creds -m SMB3 //storageN.ris.wustl.edu/WUSTL_KEY

You will be greeted with the smbclient prompt

../../_images/chpc-smb-prompt.png

II. Transfer files of interest to storageN

In the smbclient prompt, navigate to the local directory on CHPC in which your file of interest resides using lcd.

lcd /scratch/stahan

Navigate to the destination folder using the cd command.

cd Active/chpc_data

The chpc_data folder is currently empty. This can be verified with the ls command.

../../_images/chpc-ls-empty.png

Using the put command, The file can be ‘put’ from CHPC to RIS. The syntax for the put command is put <local file name> [remote file name]

put file_1.bin file_1.bin

This command will transfer file_1.bin from the CHPC scratch folder to RIS storageN and save the file as file_1.bin

../../_images/chpc-transfer.png

The 1 GB file transferred at ~ 80 MB/s and is now located in RIS storageN. This can be verified again using the ls command.

../../_images/chpc-ls-notempty.png

References