News for hackers. Stuff that matters.
Re:Files Are Fraught with Peril by greatgib on Tuesday July 23, 2019 @05:03AM (#20505386)
I did work on backup solutions and supported Dropbox, and I think that the author used Dropbox as a reference to support his own concerns about filesystems that are not related in any case to the Dropbox case.

From the filesystem point of view, I don't think that Dropbox is so much concerned about you losing your data because of corruption. As, anyway they are supposed to be safe and versioned in the cloud. And I think that their app is very simple, ie like no specific 'driver' hack.

Their real costly problem is related to metadata! All different file systems support different naming, encoding or character sets for file names. Also, they all have their specificities and limitations regarding the support of extended metadata or user right info. Also some will support file change notification or not, some will have a valid file last modification value that can have different level of decimal precision, etc...

So, as Dropbox ambition is that their app can fully backup and restore your system data, supporting backuping from one and restoring to another or the combinatory of all possible for each case is a nightmare.

Just a simple example: let suppose that you have a filename with the char "ü". With unicode normalization, this can be stored as a single character or as 2 characters "u" + ".." (letter + particule). Everyone uses the first version, but Mac HFS uses the second form. The crazy thing is that if you try to save a file with the first form in HFS, it will accept but will silently convert the filename. So, let's suppose that it was Dropbox asking to restore such a filename with the letter as a single character, later when listing your files to see if they are in sync, it will see another filename (based on filename bytes) but not the one it expects. So it might want to download it again and again and again if it was not smart.

Re:Files Are Fraught with Peril by ChrisSD on Tuesday July 23, 2019 @05:51AM (#20505545)
Yeah, the article used comments on Dropbox as a kicking off point to talk more generally about filesystem issues. The author is aware that in this specific case it was more about metadata:

> I believe their hand was forced by the way they want to store/use data, which they can only do with ext, but even if that wasn't the case...

The Dropbox examples seem to have clouded the issues actually being discussed.

Re:Files Are Fraught with Peril by aidos on Tuesday July 23, 2019 @12:45PM (#20508116)
But Dropbox doesn’t even attempt to support the more common cases (pun intended). For example, you put a file in with specific characters in upper case and it might all come out lower case on another (Linux) system.

They claim it’s because not all file systems are case sensitive, but they could at least keep the original case and use it when possible.

Then again, onedrive syncing break on a whole new level. If you upload a docx file, they’ll open it and edit the meta data so the file you added will be different from the one you get back (actually they do it to images and pdfs too).

Re:Files Are Fraught with Peril by coldtea on Tuesday July 23, 2019 @11:48AM (#20507673)
>From the filesystem point of view, I don't think that Dropbox is so much concerned about you losing your data because of corruption. As, anyway they are supposed to be safe and versioned in the cloud.

They could get to the Cloud corrupted, and then continue replicating in your devices, so no...

Re:Files Are Fraught with Peril by moolcool on Tuesday July 23, 2019 @12:15PM (#20507865)
That's where the "and versioned" comes in handy
Re:Files Are Fraught with Peril by coldtea on Tuesday July 23, 2019 @01:12PM (#20508354)
Versioned wont help if it's the first copy that got into dropbox.
Re:Files Are Fraught with Peril by greatgib on Friday July 26, 2019 @04:30PM (#20538110)
If it's the first copy, it will just be read by Dropbox. There will not be 'write' from Dropbox that can corrupt it...
Re:Files Are Fraught with Peril by greatgib on Tuesday July 23, 2019 @03:01PM (#20509256)
If it's the first copy going to Dropbox, then it is not Dropbox that wrote it and so not Dropbox that corrupted it by writing in a wrong way...
Re:Files Are Fraught with Peril by j4_hnews on Tuesday July 23, 2019 @09:25AM (#20506518)
Also to note, Dropbox was likely inspired by the open source Unison file-synchronization tool. So they have to handle all of the shortcomings and caveats that Unison does not handle out of the box:

Caveats: https://www.cis.upenn.edu/~bcpierce/unison/download/releases...

Symbolic links: https://www.cis.upenn.edu/~bcpierce/unison/download/releases...

Permissions: https://www.cis.upenn.edu/~bcpierce/unison/download/releases...

Cross platform issues (unsupported characters across operating systems and file systems): https://www.cis.upenn.edu/~bcpierce/unison/download/releases...

Edit: Inspired by the the Unison Spec (published on 2004) which may be why we have conflict files etc. when using Dropbox http://www.cis.upenn.edu/~bcpierce/papers/unisonspec.pdf

Re:Files Are Fraught with Peril by hemancuso on Tuesday July 23, 2019 @10:02AM (#20506783)
Fairly sure this is not true, unless you have a source. Drew did all the client side work in python/librsync to get going.
Re:Files Are Fraught with Peril by j4_hnews on Tuesday July 23, 2019 @10:42AM (#20507083)
Ok, may have been inspired by. Here's some relevant backstory from 2011: https://www.wired.com/2011/12/backdrop-dropbox/