Skip to content

Instantly share code, notes, and snippets.

@iwconfig
Created October 7, 2025 10:47
Show Gist options
  • Select an option

  • Save iwconfig/af302e4ec3cd08241bef17b6c998bc96 to your computer and use it in GitHub Desktop.

Select an option

Save iwconfig/af302e4ec3cd08241bef17b6c998bc96 to your computer and use it in GitHub Desktop.
run rclone copy/sync/whatever in batches of ≤ N files
# batches of 200 with a ~5-180 seconds delay
# the `cat "{}" >/dev/null &` pre‑caches the vfs, reducing telegram io load and speeding rclone.
find /some/dir -type f -printf '%P\n' -exec sh -xc 'cat "{}" >/dev/null &' \; |
xargs -n200 -d '\n' bash -c '
printf "%s\n" "$@" |
rclone -v sync --check-first --transfers 4 -PM --files-from - \
/some/dir/ some-remote:some/remote/dir/ --error-on-no-transfer &&
{ s=$((RANDOM % 176 + 5)).$RANDOM; echo sleeping $s seconds; sleep $s; }
' _
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment