News for hackers. Stuff that matters.
Re:Ask HN: Is there a data loss bug lurking in MS365 backup ... by prmph on Wednesday September 06, 2023 @02:51PM (#37409396)
I wouldn't trust any MS cloud product to not lose my data.

A year or two ago I lost data backing up my OneDrive files to my local HDD. I was told my backup had succeeded, and so I deleted the files on OneDrive. Later, when I tried to extract some large files from the backup, I saw that they had been replaced with some text files, containing a message that the download of those files failed.

What the heck... Was I expected to extract the archive and go through all the files one-by-one (there are thousands of them) to check if every file was properly backed up?

Re:Ask HN: Is there a data loss bug lurking in MS365 backup ... by grepfru_it on Wednesday September 06, 2023 @03:15PM (#37409774)
>I wouldn't trust any MS cloud product to not lose my data.

I raised issues of data loss and management prioritized shipping to customers instead. I can't say I blame them, they can take those risks knowing customers will accept them because their big name is behind the product. That said, I heard the issues were resolved so proceed with caution

Note: this was not onedrive

Re:Ask HN: Is there a data loss bug lurking in MS365 backup ... by ansible on Wednesday September 06, 2023 @03:18PM (#37409815)
> later, when I tried to extract some large files from the backup, I saw that they had been replaced with some text files, containing a message that the download of those files failed.

That's a really asinine failure mechanism. At the very least, it should create "filename-FAILED_TO_RESTORE" or something like that.

Something that I was instructed to do nearly 30 years ago when at Motorola was: never overwrite the original files. Standard procedure was to make a directory called "BACKUP" (or "RESTORE"?) and put the files in there instead. The requesting person was then responsible for moving the files back to their original location, so any screwups were on them, not us (IT).

Re:Ask HN: Is there a data loss bug lurking in MS365 backup ... by justsomehnguy on Wednesday September 06, 2023 @04:40PM (#37410989)
The dogfooding was a thing 30 years afo.
Re:Ask HN: Is there a data loss bug lurking in MS365 backup ... by ThePowerOfFuet on Thursday September 07, 2023 @04:20AM (#37416121)
Sun did it much more recently than that.
Re:Ask HN: Is there a data loss bug lurking in MS365 backup ... by bobbylarrybobby on Wednesday September 06, 2023 @05:05PM (#37411307)
I think the only correct way to handle failures is to have a single file called ONE_DRIVE_RESTORE_FAILURES.txt at the restore destination root, and in it list the path to every file that failed to download, one path per line. Much harder to miss than individual files with conspicuous names.
Re:Ask HN: Is there a data loss bug lurking in MS365 backup ... by bombcar on Wednesday September 06, 2023 @02:58PM (#37409515)
This is exactly what I've encountered with all the cloud-based file storage apps now that they've gone to "dynamic download".

You will get failed downloads quietly.

Re:Ask HN: Is there a data loss bug lurking in MS365 backup ... by ryan87 on Wednesday September 06, 2023 @03:18PM (#37409824)
> What the heck... Was I expected to extract the archive and go through all the files one by one (there are thousands of them) to check if every file was properly backed up?

There are a lot of silent pitfalls like that in my experience. For example, if you use folder level encryption with Synology Active Backup for MS365 it can silently mangle the file names due to path length restrictions. You'll end up with files that have "file name too long" as part of the file name.

That's why I'm trying to reconcile every file in this data set. I don't trust anything without being able to personally verify it at this point.

Re:Ask HN: Is there a data loss bug lurking in MS365 backup ... by bob1029 on Wednesday September 06, 2023 @04:12PM (#37410644)
> For example, if you use folder level encryption with Synology Active Backup for MS365 it can silently mangle the file names due to path length restrictions.

Is this Microsoft's fault or Synology's fault? Much of this discussion sounds like an inability to distinguish between first and third party products.

Re:Ask HN: Is there a data loss bug lurking in MS365 backup ... by ryan87 on Wednesday September 06, 2023 @04:36PM (#37410942)
> Is this Microsoft's fault or Synology's fault?

It's 100% on Synology.

> Much of this discussion sounds like an inability to distinguish between first and third party products.

Yeah. Did I explain it badly or something? I thought I made it clear that I'm having an issue with 3rd party backup solutions and that it's possible they're using some shared API from Microsoft, but most people seem to be focused on OneDrive specific issues.

Re:Ask HN: Is there a data loss bug lurking in MS365 backup ... by hulitu on Thursday September 07, 2023 @05:04PM (#37425575)
> Is this Microsoft's fault or Synology's fault?

It it were Microsoft's fault the filenames would have bin filena~1, filena~2 etc. /s

Re:Ask HN: Is there a data loss bug lurking in MS365 backup ... by dosshell on Wednesday September 06, 2023 @03:09PM (#37409689)
Does someone know if rclone also can fail silently when syncing onedrive? [0]

From my experience it doesn't but onedrive provides, sometimes, filesizes that are wrong, so checking file integrity is a tiny bit more of a pain.

[0] https://rclone.org/docs/