News for hackers. Stuff that matters.
Re:OpenSSH 9.0 by paulmd on Friday April 08, 2022 @01:53PM (#30960330)
the biggest weakness of rsync is that it's single-threaded. Single-thread-copy with millions of small files is painfully slow, Rsync is as good as it can be but you just need more threads.

I think that's what a sibling is getting into with the "better to tar files and send them over ssh in some cases" thing. And yes, you can hack it in after-the-fact with xargs/etc but it's clunky compared to just having native multithreading like rclone/etc.

Re:OpenSSH 9.0 by jimpudar on Friday April 08, 2022 @02:57PM (#30961111)
If you use GNU Parallel, there's a fairly easy way to parallelize [1].

That being said, I hadn't heard of rclone - thanks for mentioning it, it looks amazing. I'll definitely be trying this out for my use cases...

[1] http://www.gnu.org/software/parallel/man.html#example-parall...

Re:OpenSSH 9.0 by robertlagrant on Friday April 08, 2022 @05:01PM (#30962465)
Wow, Parallel also looks awesome. Never seen it before.