Rsync Command Cheat Sheet

common-media-sync.md

Quick Adds

• Direction: Pull
• User: inanimate
• Remote module name <mod>/<path>/*
• Enabled
• Recursive
• Times (reconsider!)
• Archive
• Preserve Extended Attributes
• Delay Updates
• -Pvihu --timeout=300 --chmod=ug=rwX,o=rX --ignore-missing-args
• (test first!) --dry-run
• (if needed) --ignore-existing
• (if needed) --log-file=/tmp/rsync.log
• (if needed) --size-only, --no-times

rsync \
  --archive \
  --xattrs \
  --update \
  --itemize-changes \
  --delay-updates \
  --partial \
  --progress \
  --human-readable \
  --verbose \
  --timeout=300 \
  --chmod=ug=rwX,o=rX \
  --dry-run

Optionally, add --ignore-existing if copying from NTFS to a "normal" filesystem.

I've added --ignore-missing-args as a needed flag to ensure when using a wildcard derp/* for the source, but which nothing exists inside of derp, rsync doesn't throw an error.

A Note on --size-only

If the modification time of the source file has changed, rsync will re-transfer the file >f..t......

In example, if you copy data from one filesystem to another with a standard cp -r command, it will not preserve the modtimes. Hence when an rsync runs against the new location, it will trigger a transfer of the file again.

To avoid this, you can use --size-only to ensure rsync only considers the size of the file and NOT the modification time when considering if it should transfer. Rsync will still update the modtime on the destination though. This should only be used for certain circumstances so please consider your use case.

Note: rsync will still update the modtime on the destination files .f..t......! To avoid this, use --no-times!!

In the future, consider using cp -a or a simple rsync -aHAX to transfer from one local filesystem to another.

Syncing Backup locations

Assuming you backups on the host itself, which its managing/trimming, etc..

Everything from the above but do the following:

• Leave out --ignore-missing-args
• Leave out -u (update), see below
• Consider leaving out the chmod, permissions don't really matter.
• Include --delete-delay or at least -delete (tickbox) - depends on where trimming is being done!

rsync-local-and-network-reference.md

rsync --dry-run -auhviPHX /mnt/bar /mnt/foo/ explainshell

*This will result in bar existing in foo so you'd have /mnt/foo/bar/bar1.jpg

rsync --dry-run -auhviPHX /mnt/bar/* /mnt/foo/ explainshell

*This will result in solely the non-hidden contents of bar (bar/*) existing inside foo so you'd have /mnt/foo/bar1.jpg

Network Transfer Considerations (tailscale/sshfs/etc)

When using this to transfer over a network, consider using the following:

• --delay-updates which will ensure your active users aren't seeing temp/stub files
• --timeout=300 has rsync quit out if the connection is failing
• When referencing a module + path, to copy the contents of the directory, verify you use <module-name>/<dir>/
• To view modules or their contents, use rsync rsync://<hostname>/ or rsync <hostname>::, then build the path to view its contents.

Don't use compress! You don't need it and you'll incur extra cpu overhead which will impact performance

Setting Permissions for Destination

Use the --chmod=ug=rwX,o=rX flag to rsync to ensure 664 for files and 775 for directories (if you want group and user to have same permissions).

This executes post --perms (inherently included in --archive). If using --no-perms, it won't even consider the source perms, defaulting to the standard perms for the user but still executes the --chmod post transfer either way so you get the same result.

Handling Destination

Generally, you'll more than likely want --update for most cases unless you want a perfect 1:1 mirror (i.e. source is authoritative)

• --ignore-existing - Skip all existing files (never updates or overwrites them).
• --update (-u) - Skip files that exist and have a newer timestamp on the destination (file size not considered, still updates older).
• --existing - Only update files that already exist (ignores new files on the source).

Note: This might not work as expected for filesystems that handle timestamps differently, cough..NTFS

Man Pages

https://linux.die.net/man/1/rsync https://www.man7.org/linux/man-pages/man1/rsync.1.html

rsync-mirrors-reference.md

rsync --dry-run -ahviyHP --delete-delay --delay-updates --timeout=300 rsync://mirror.rit.edu/ubuntu-releases /mnt/isos/linux/ubuntu-releases explainshell

This will result in /mnt/isos/linux/ubuntu-releases receiving the contents (i.e. 24.10/, xenial/, etc.) of the module!

-a is the same as -rlptgoD

-P is --partial and --progress - just re-execute command if it times out!

Slashes and Wildcards Matter when defining the Source!

• /source/* (Preferred!) → only the non-hidden contents of the folder are copied
• /source/{*,.[!.]*,..?*} → all contents, including hidden, of the folder are copied
• /source/ → contents of the folder are copied, including the timestamp and permissions of the encasing folder source to the destination containing folder destination/
• /source → folder itself is copied into destination

This is different with referencing a rsync module! It will always sync the contents of the module, not the module name as a directory.

Example

rsync -a /mnt/usb1/data/ /mnt/pool/backup/     # Copies contents
rsync -a /mnt/usb1/data  /mnt/pool/backup/     # Creates /backup/data/

Bandwidth Limit in kbps

--bwlimit=25600 # ~ 25MBps

Any matches glob

--exclude='*live*'

Specific path

--exclude='somefolder/README.txt'

Specific top Directory

--exclude='temp/'

Exclude everything under a path (incl subdirs)

--exclude='temp/**'

Exclude all "thumbnails" directories matching anywhere

--exclude='*/thumbnails/'

Exclude by file extension

--exclude='*.mp4'