Re:No Space Left on Device
by jobigoud on Tuesday September 06, 2016 @02:42AM
(#12434013)
I came to the same conclusion about sharding files from the get go. I had a similar experience where my host wouldn't accept to synchronize a directory with more than 5000 files in it. (Software producing the files is my own note-taking application where each page is stored as an individual file to speed up backup of only the pages that changes).
[ Parent
| on HN ]
Wouldn't 5000 files of note-taking easily fit into RAM? Why bother dealing with thousands of small files? Also, rsync does quite a good job with intransferring multi-MB (or even larger) files efficiently while copying only what's actually changed.
I feel more secure by dealing with many individual files. For example in case of a conflict during sync, I can visually diff between the two pages very easily. This happens once in a while. Also for example if more than 5% of files have changed, the sync is stopped and asks for manual confirmation to prevent catastrophies.