Re:Tell HN: macOS Ventura broke the Raspberry Pi Pico ecosystem
by uranusjr on Thursday December 08, 2022 @05:26AM
(#33906365)
I don’t understand any of the technical details involved here, but reading your comments makes me seriously wondering, does RPi rely on these everywhere, or just when plugged to a Mac? Because all three things seem like pretty reasonable optimisation opportunities for a file manager to implement, and given how USB storage devices have been around I’m very surprised if they managed to get away without issues until now.
Re:Tell HN: macOS Ventura broke the Raspberry Pi Pico ecosystem
by devanl on Wednesday December 07, 2022 @11:41PM
(#33904243)
In case you haven't seen the UF2 file format, it's specifically designed to be robust w.r.t ensuring that the content makes it to the target intact despite common OS quirks.
https://github.com/microsoft/uf2
- Data is divided into 512 byte blocks, which lines up with the mass storage transfer size.
- Each block has an identifying magic sequence, a payload address, and size, which can be used to identify the destination.
- This allows the emulated storage to ignore metadata blocks and to handle out-of-order and repeated blocks.
Re:Tell HN: macOS Ventura broke the Raspberry Pi Pico ecosystem
by catiopatio on Wednesday December 07, 2022 @11:48PM
(#33904285)
The UF2 format is a hack that can be made to generally work (e.g. it relies on things like per-block beginning and ending magic numbers to identify blocks that should be written to flash, which means isn't dependent on seeing FAT table writes in a particular order).
UF2 is just the file format, though. The mechanism for reading/writing UF2 files is via the emulated USB mass storage, which is fundamentally broken.
The USB storage device responds to SCSI write commands with a success status, but actually throws away data, such that a successive read does not return the previously written blocks.
To an OS or filesystem, that's just a failing drive. You might get lucky and things might work.
The only way this ever worked was pure luck, relying on the OS:
- Writing file blocks in the desired order — which is not guaranteed.
- Writing filesystem metadata in the desired order — also not guaranteed.
- Not caching any important pages, so that the VM page cache isn't out-of-sync with the block device after it silently discards writes.
- Not caching any important filesystem metadata, so that the filesystem implementation isn't out-of-sync.
I'm surprised (1) anyone thought this was anything other than a risky hack, and (2) not only did RPi actually ship this broken design, but did so in ROM, and thus cannot patch it.
[edit]
Surprised at the downvotes for posting a technical rebuttal to an inflammatory comment.
As someone with a few decades in OS/kernel development, this doesn't seem like a controversial or even difficult question. The implementation is broken and cannot work reliably.
If you disagree, then please explain why you believe the USB mass storage emulation is not broken.