News for hackers. Stuff that matters.
Re:Windows is the strangest, or hardest, operating system to... by trelane on Wednesday November 30, 2022 @10:00AM (#33801635)
I still don't understand the loss. The original data that was going into the zip was still there. Or did you the write pipeline delete it at the end of something?

More precisely, what data was lost, and how did it get lost?

Re:Windows is the strangest, or hardest, operating system to... by scbrg on Wednesday November 30, 2022 @10:21AM (#33801930)
There are three processes involved.

Process P, the data producer, writes to file foo.

Process Z, the zip-process, reads file foo and produces foo.zip

Process rm, removes file foo.

I imagine it went a bit like this:

    P --output foo &
    Z < foo > foo.zip
    rm foo
    # time passes
    # process P terminates
Any data written to foo since Z was run is now lost.

User 323 assumed that either rm would fail, letting them know that foo cannot be deleted, or that P would throw an error when the file was deleted.

(I don't think this is a reasonable expectation, and it's a failure on 323's hand of not learning the UNIX file model, but there's the situation).

Re:Windows is the strangest, or hardest, operating system to... by trelane on Wednesday November 30, 2022 @10:35AM (#33802131)
In this case, it would have been deleted anyway.

I think I can see the loss:

1. Start zip. 2. In parallel, copy the intermediate zip file. 3. Delete the zip 4. Delete originals

The data was deleted in step 4 and is the loss. It notably would also had occurred had the zip terminated early or not zipped _all_ the files for some reason. The process was not resilient.

Re:Windows is the strangest, or hardest, operating system to... by 323 on Wednesday November 30, 2022 @10:53AM (#33802478)
I understand the UNIX file model. My mistake was that I thought that the data producer was writing to a different file (think something like log-rotation).

I think it's debatable if it's reasonable to continue allowing writing to an unlinked file.

There was discussion some time ago about Postgres data loss because of something similar, where the linux kernel returned successfully from fsync despite the movable media not being present anymore.

> How is it possible that PostgreSQL used fsync incorrectly for 20 years, and what we'll do about it

https://archive.fosdem.org/2019/schedule/event/postgresql_fs...