The questions stands; how would it PXE?
All of them now have a very specific model industrial MicroSD card in them, and none have lost or corrupted data, or refused to boot, since those cards were added. They had SanDisk ultra cards before that, and they had to be reinstalled all the time.
I picked the card because it's an aMLC* model (cheap overall, $15 for 4GB, $25-28 for 8GB), which significantly improves write endurance even compared to MLC cards, and because the controller is designed to handle sudden power loss, which is where a lot of these Pi+SD problems come from; consumer MicroSD cards are generally designed for use in devices that have batteries and rarely if ever lose power suddenly, so the controller can take risks for optimization that it wouldn't otherwise want to take.
They're also rated down to 0C, which I can say is definitely true as I have some operating close to that right now.
This is the card: https://www.digikey.com/product-detail/en/atp-electronics-in...
4GB isn't huge in a world where 32/64GB cards are common for a similar price, but a full headless OS installation doesn't even take half of that, and there's no real use for more space most of the time.
* with aMLC mode, it's still MLC flash (so cheaper than SLC), but each cell only stores a single bit rather than 2 (MLC) or 3 (TLC), so there are only 2 voltage states it needs to be able to measure, rather than 4 (MLC) or 8 (TLC). That means you lose some storage capacity (half vs MLC), but the tolerances for the voltage measured in each cell is significantly relaxed, so it remains usable long past the point where a plain MLC mode card would not be able to accurately program and measure the voltages that represent the 2-bit values of 00, 01, 10, or 11 anymore.
Plus, it costs $3 each, is stamp-sized and has Wifi.
The ESP32s were routinely dropping from WiFi (some being crashes, some just disassociating and never connecting again). ESP-IDF has had quite a few WiFi fixes, but it's difficult to guarantee that your own code isn't directly causing the problem (which you get for free running in userland on Linux), and dealing with it was becoming a huge time sink.
Most of the Pi Zero W boards I have were $5 at Microcenter, the MicroSD adds to the cost a little but they're extremely reliable as a result, and my own code is no longer running in the same address space with the scheduler and WiFi driver under extremely tight memory constraints :)
the ESPs are most definitely better for higher volume solutions
EDIT : now that you mention it - micropython is pretty cool -- and I haven't used it - need to try that on the ESP :)
If you then adjust your systemd journal to not write very often you have eliminated most of your SD writes.
The remote sensor systems I run lose power from time to time during the year, sometimes once a day. I’d say each time there used to be something like a 0.1% to 1% chance of corruption. Since I’ve moved to /run and a slower journal I haven’t lost an SD card[1]. Thus far it seems I have at least an order of magnitude improvement.
[1] Well, I had a wind turbine tower collapse in an inaccessible location. The raspberry pi is in a lake now, but it can’t have gone far so I don’t call it “lost”. I’ll dive and recover it in the spring. That camera/sensor has spent a couple months underwater before. In fact, I’ve had three systems end up underwater for months and all the SD cards have been fine when recovered.
> If you then adjust your systemd journal to not write very often you have eliminated most of your SD writes.
The trouble is that the OS isn't the only one writing to flash in the card, the MicroSD card controller is erasing and writing to it at random times as well even if the filesystem is read-only, with no coordination between it and the OS.
As a result you can easily end up in situations where the card decided to run some housekeeping to manage the flash just milliseconds before a power cycle or a power loss, and seemingly all consumer MicroSD cards are not up to the task of keeping the data intact at the same time.
I ended up creating a RAM disk and only writing to that. Once every 5 minutes or so I would flush anything that needed to be permanent to a partition on the SD card. And once every hour I'd save anything really important to an s3 bucket. SQLite also performed much better running in memory than it did directly from on the SD card.
That mostly solved the problem, but I still shipped every unit with an extra SD card and instructions on swapping it out just in case.
Raspberry Tau comes to mind. (https://en.wikipedia.org/wiki/Turn_(geometry)#Tau_proposals). It could mean just two Raspberry Pis, or I guess any amount more than one.
Mounting root read only and only writing explicitly through the "ro/re" scripts helps a lot and should be the default IMO.
The biggest issue some people have with RPis though, is their hackish software, e.g. the kernel. This has become better apparently, but it always was a shame that you couldn't just run a generic kernel/distro.
I had good experiences with the BeagleBoneBlack, they come with eMMC and quite a few smart additions (e.g. a power button). Twice as expensive though.
We're using hundreds of raspberry's as digital signage devices and before deploying them I checked a few of them and found out that sometimes the source files of the app were empty.
After a lot of headscratching and googling we boiled it down to a write delay to the SD card.
If you do echo "test" > test.txt and power off the device like 5 sec. after, and power it on again, there was a file called test.txt with 0 bytes.
I'm still baffled how something like this can be a common problem especially since raspberry pi usage is so widespread
* The OS is is always read-only. While this doesn't prevent all possible corruption scenarios, it's pretty close and you can be quite sure the system always boots.
* System updates are implemented as A/B. So even during updates, the OS the system booted from isn't touched.
* All content data is on a third partition and together with the OS files is constantly checked for indications of corruption.
* If the OS detects any bit error on the OS files, it is reinstalled (using the A/B mechanism above)
* If there's a bit error on any of the content files, they are deleted and downloaded again
* If the content filesystem is broken beyond repair the system reformats everything and restores everything automatically by redownloading
* Various watchdogs and checks constantly monitor system health and notify you on the dashboard.
The service has multi-million hours of service by now and the only case of filesystem corruption was due to a really crappy SD card. When done properly, the Pi can be reliable. That's also the reason why there is no way to properly shut down the system. Just unplugging power shouldn't be an unexpected circumstance but something you optimize for. Starting with a normal Raspbian and expecting that to work is a bad idea.
It always looked to me like the Pi was supposed to be used some steps before hardware integration and shipping.
I am glad products are built on it but I was a bit worried when I had to evaluate a Pi based solution for digital signaling last year. Maintenance was a key factor in the proposals.
Most file system drivers delay writes, I suspect some variant of your test (if it was unsafe shutdown) could be reproduced on almost any system.
Also raspis do NOT like big fast SD cards with tight tolerances. I've had the best luck with old class 4 cards. Class 10 cards don't last nearly as long with regular power cycling.
The options are the third column in your /etc/fstab file. For example, if you had "errors=remount-ro", you'd change it to "errors=remount-ro,sync".
(Or is the Raspi problem actually in hardware?)
https://utcc.utoronto.ca/~cks/space/blog/unix/TheLegendOfSyn...
Except that you don't need to buy an SD card. Or a special 2.5A power supply. And you can access the it solely over USB so you don't need to hook up an external keyboard, mouse, and HDMI to get it running headless. And you can actually get the chips and their datasheets for the BBB. etc.
But, hey, the RPi is half the price, man.
You can also PXE boot without an SD card, which may be best for file system durability.
So, if you're overclocking, don't do that if you're using SD cards.
With Pi firmware newer than late 2016 (if I recall) you can write a file to an SD card that sets a bit in NVRAM somewhere telling it to boot to USB from that point onwards, and then you can operate without that SD card or any other going forward. Thumb drives are usually faster and more durable than SD cards.
Been running daily for a couple of years now as a Pi-hole with no SD corruption (3B+ model). Still using the same card as I had in the beginning as well (Samsung Evo)
Even many wall chargers don’t provide this - the standard one shipped with an iPhone that many people likely have isn’t powerful enough either.
I think continuous 2.5A is not needed, but is needed for an instant or two, here and there, if you are running headless.
All most people would need is a sizeable capacitor across the input power pins, perhaps.
Power supplies (people buying cheap ones, or being supplied with cheap ones -- looking at you, Amazon raspi kit sellers) are the number one source of failures that I have encountered with the pi.
A good power supply is a lot more expensive than a cheap one, and people are reluctant to invest in a $20 supply when a $5 one has the same rating, so I can't blame anyone for that other than the sellers.
That said, I do agree with OP's dislike of SD cards. I've had more than my share go bad in other devices.
I had some unstable power (not affecting any other home electronics) and was looking in some UPS hat but opted to go security DVR way. Bought 7Ah battery and 70W buffer PSU[1] and DC-DC 12V to 5V supply[2].
Could not be more happy. Random problems with SD cards are gone. Only thing to add is some battery voltage sensing to gracefully shutdown RPi in case of longer power outage.
Thus, I can tell that many problems with RPi may come from inadequate PSU.
1. I have similar https://www.meanwell.com/webapp/product/search.aspx?prod=SCP... 2. https://www.meanwell.com/webapp/product/search.aspx?prod=SD-...
Pi 3 B has support for both USB and network booting in its bootloader, but they are disabled by default. They are turned on by modifying some OTP configuration bits, which is a permanent change. Once set the SD card is no longer required.
Pi 3 B+ has network and USB booting enabled by default.
Presumably Pi 3 A+ will support USB booting out of the box, who knows it it'll also support USB ethernet adapters for PXE.
Not true anymore.