Data Transfer using rsync

Owned by Jodie Sprouse
Last updated: Oct 16, 2024 by Meghan Netterville
1 min read

Take a look at: CAC Virtual Workshop for Data Transfer: rsync

i.e. to copy from your campus laptop to your home directory on the BRB cluster:

  1. If you are not on campus, connect to the Weill VPN.

2. We recommend one uses screen program to conduct the copy in the background:

  • open terminal window and type:

* At the prompt 'press space' or 'return to end': press space (you remain in screen)
* To exit screen at any time, type: exit

  • Detach from the screen session: press CTRL + A, then D

  • Resume your screen session: screen -r

Note: screen has many options to explore. The above should get you started.

3. rsync -av laptopFromDir/ loginid@scu-login01.med.cornell.edu:~/loginToDir/

 

Note: Be careful with or without the slashes using rsync. The above states "copy everything found under laptopFromDir to my home directory in a new directory called loginToDir that I previously created.

 

  1. Copy Files from Remote to Local

    1. rsync -av loginid@scu-login01.med.cornell.edu:~/loginToDir/ laptopFromDir/

  2. Copy a Directory Locally

    1. rsync -a /path/to/source/directory/ /path/to/destination/directory/

 

 

rsync is a powerful tool with many useful options. Here’s a quick list of examples:

  1. -a: Archive mode, preserving symbolic links, devices, attributes, permissions, and ownership.

    rsync -a /path/to/source/ /path/to/destination/directory/

  2. -v: Verbose mode, providing detailed information during the transfer.

  3. -r: Recursively syncs folders.

  4. -nv: Performs a verbose dry run, showing what the command would do without actually making any changes.

  5. -z: Compresses the data during transfer to save bandwidth.

  6. -P: Combines --partial and --progress, keeping partially transferred files and showing progress during transfer.

  7. --delete: Deletes files in the destination that are not present in the source, ensuring the destination is an exact mirror of the source.

  8. -t: Preserves modification times of files.

  9. --exclude: Excludes files matching specified patterns from the transfer.

  10. --include: Overrides exclusions for specific patterns.