News for hackers. Stuff that matters.
Re:New Raspberry Pi Compute Module 3+ by StavrosK on Monday January 28, 2019 @06:14AM (#19016563)
This looks delectable for the eMMC storage alone. I wish there were a Raspberry Pi model that came with it... I'm tired of SD cards causing problems and eating my data or (more frequently) causing the Pi not to boot.

I also wish Raspbian had an option to mount all the write-heavy directories (logs, etc) on tmpfs, so they wouldn't wear the card out so quickly.

Re:New Raspberry Pi Compute Module 3+ by blihp on Monday January 28, 2019 @06:32AM (#19016655)
Well it's just Linux so you definitely can and that's what I do. As a result I've been running my pi 2 on the same card for nearly 4 years now with no issues. But if your point is that it should do it out of the box for you, I agree it shouldn't require manual setup.
Re:New Raspberry Pi Compute Module 3+ by StavrosK on Monday January 28, 2019 @06:47AM (#19016727)
Yeah, that's what I meant. It's not hard to do, but it's one more thing and it would be nice if it were done automatically.
Re:New Raspberry Pi Compute Module 3+ by naikrovek on Monday January 28, 2019 @10:40AM (#19018258)
I believe recent versions of Raspbian (at least the ones available on raspberrypi.org) mount all of /var as tmpfs. That's a RAM filesystem.
Re:New Raspberry Pi Compute Module 3+ by StavrosK on Monday January 28, 2019 @10:46AM (#19018313)
That's great! I'll need to rebuild my Raspberry Pi if that's the case, maybe make everything run in some repeatable way so I can reprovision quickly if I need to, thank you.
Re:New Raspberry Pi Compute Module 3+ by arthurfm on Monday January 28, 2019 @07:34PM (#19022616)
Re:New Raspberry Pi Compute Module 3+ by dividuum on Monday January 28, 2019 @06:21AM (#19016601)
When done properly, SD cards are reliable enough. For my digital signage product the custom OS is highly optimized around avoiding IO as much as possible. There are thousands of devices, some running more than 4 years now, without any SD related problems.

I also really like the fact that the Pi is pretty much stateless. So switch SD cards and you can be mostly sure everything still works. If storage is bundled, that's a lot more difficult.

Re:New Raspberry Pi Compute Module 3+ by sgt on Monday January 28, 2019 @07:22AM (#19016912)
For most use cases, SD cards on Raspberry Pi's are not reliable enough.

I realize you may be successful in running signage devices, but most people want to simply be able to run common class 10 card (e.g. from Verbatim, Kingston, Sandisk), using a standard OS setup and leave the Pi running without having it corrupting your files. This simply isn't possible, in my experience. Also look at the OpenHAB forums as a reference.

The more regular disk activity, the worse it gets.

We went this route in our office and every 3-6 months we had corruption on each of many dozen Pi's. Once we switched to using USB drives (spinning or SSD) the issues were sorted out.

Re:New Raspberry Pi Compute Module 3+ by mschuster91 on Monday January 28, 2019 @09:08AM (#19017485)
> but most people want to simply be able to run common class 10 card (e.g. from Verbatim, Kingston, Sandisk), using a standard OS setup and leave the Pi running without having it corrupting your file

To be fair: try the same with cellphones. I have a shitload of micro-SD cards gone bad on me, and yes genuine SanDisk not cheap fakes off of Amazon.

microSD cards are the cheapest crappy flash chips you can get with a microcontroller strapped in front for error correction.

Re:New Raspberry Pi Compute Module 3+ by H1Supreme on Monday January 28, 2019 @10:32AM (#19018176)
I don't even put SD cards in my phones anymore due to corruption. I always got the best SanDisk's I could find. Luckily my Pi that runs probably 8 hours/week hasn't failed, but I have the drive imaged just in case.
Re:New Raspberry Pi Compute Module 3+ by bradfa on Monday January 28, 2019 @08:55AM (#19017403)
Cheap SD cards are going to return less than desirable results. Higher quality cards such as SanDisk Ultra Mobile or Samsung Pro series should last quite a while when used as a normal "disk" on a system like this. As an added benefit, using a card like a Samsung Pro will also get you MUCH better file system speed for both reads and writes on common Linux file systems like ext4. The controller inside a Samsung Pro series card is much more advanced than run of the mill cheap Kingston cards.

If you really need durability, there's also a handful of SLC (single level cell) SD cards on the market which will last an order of magnitude longer in terms of writes as compared to consumer level MLC/TLC cards. SLC cards aren't cheap but you get what you pay for.

Re:New Raspberry Pi Compute Module 3+ by monocasa on Monday January 28, 2019 @10:45AM (#19018310)
> Higher quality cards such as SanDisk Ultra Mobile or Samsung Pro series should last quite a while when used as a normal "disk" on a system like this.

Nope, it's mainly marketing. Cards from those brands are fundamentally not great for sub million unit purchases. We tried to use some for the root fs for a fairly high availability appliance. If you don't want one in twenty going out in a month (and we figured out a way to make them go out after 8kb/s for 8 hours with a specific access pattern without power drops), then you need 'industrial SD cards'. They'll treat you like adults when it comes to support, will have lot codes that actually map to internal changes, etc. Trying to sell a product with consumer SD cards (yes, even the ones that call themselves 'Pro') is basically a futile exercise.

Re:New Raspberry Pi Compute Module 3+ by StavrosK on Monday January 28, 2019 @07:03AM (#19016823)
Sure, but I don't have the time to highly optimize a custom OS just to put up an MQTT server in my parents' house. I want to do the work once and have it Just Work after that.
Re:New Raspberry Pi Compute Module 3+ by skl_ on Monday January 28, 2019 @06:28AM (#19016632)
Out of interest, how did you approach disk encryption? That would be my concern with removable storage. The other thing I'm keeping an eye on is Arm's LittleFS for storage longevity.
Re:New Raspberry Pi Compute Module 3+ by dividuum on Monday January 28, 2019 @06:36AM (#19016671)
It's possible to write 256 bit of user data to the Pis internal OTP memory. You can store a non-changeable key there. Of course that's then easily readable if you can run execute commands on the Pi.

LittleFS (recently posted here) is indeed interesting, although from the documentation I couldn't figure out

1) if it works for larger devices as it seems it seems mostly intended for sizes of a few MB. I might be totally wrong with that.

2) if/how it handles corruption of complete blocks(?) of SD memory. As far as I understand, an SD card doesn't necessarily use 4K block sizes and corruption might hit multiple blocks at once. If LittleFS stores both previous and next version next to each other, I don't think that'll help in that case.

Re:New Raspberry Pi Compute Module 3+ by skl_ on Monday January 28, 2019 @07:23AM (#19016918)
Oh, there's an OTP in the MPU? I didn't know that.

Yes with 1) I'm not sure either but there's some sporadic discussion in the Github issues with various attempts. From what I remember certain scenarios currently cause all blocks to be re-scanned which will obviously get slower with capacity.

With regards 2), I was under the impression each block is allocated according to the wear-levelling algo[0] so I don't think they're contiguous if that's what you mean. Also block size is configurable.

I tried implementing LittleFS on a 32MB SFDP flash with Mbed but couldn't get any decent performance out of it. It kept spewing out "bad block" errors an re-allocating everything. I hope it's a more viable option in future though as I do like the design.

[0] https://github.com/ARMmbed/littlefs/blob/master/DESIGN.md

Re:New Raspberry Pi Compute Module 3+ by opless on Monday January 28, 2019 @09:32AM (#19017634)
I posted it (though I'll freely admit hackaday was the source that I discovered it through)

Personally I want to use it as an internal filesystem for various projects (like games - instead of zip/pak files, which are really read only)

even littlefs over https might be an interesting naval gazing project :D

Re:New Raspberry Pi Compute Module 3+ by Abishek_Muthian on Monday January 28, 2019 @06:56AM (#19016780)
I store the data in encrypted USB flash drives & store the key in eCryptfs on the SD card.
Re:New Raspberry Pi Compute Module 3+ by forinti on Monday January 28, 2019 @08:57AM (#19017412)
You can (should) disable logging on raspbian:

  sudo systemctl stop rsyslog

  sudo systemctl disable rsyslog
It also makes sense to create a temporary directory in memory, if you don't need much space.
Re:New Raspberry Pi Compute Module 3+ by StavrosK on Monday January 28, 2019 @09:05AM (#19017463)
That's not a great solution because I do need logging sometimes, especially to see what processes are doing, I just don't need it persisted.
Re:New Raspberry Pi Compute Module 3+ by jasonjayr on Monday January 28, 2019 @09:25AM (#19017589)
Unless you set the option in journald.conf systemd/journald will retain logs in an auto-rotated tmpfs.

It does this whether or not a syslog daemon is running. (All messages it captures goes through the journal and then are forwarded along)

You'll still be able to look @ recent logs with journalctl commands.

Re:New Raspberry Pi Compute Module 3+ by opless on Monday January 28, 2019 @10:42AM (#19018279)
This is what I do for an IR camera project.

Set up everything how I like it, set root read-only and mount tmpfs in various areas such as /var/log etc..

Re:New Raspberry Pi Compute Module 3+ by StavrosK on Monday January 28, 2019 @09:34AM (#19017642)
Oh really? That's awesome then, thanks!
Re:New Raspberry Pi Compute Module 3+ by jsight on Monday January 28, 2019 @11:01AM (#19018458)
I've heard that these are better: https://www.digikey.com/product-detail/en/atp-electronics-in...

I haven't tried them myself, though.

Re:New Raspberry Pi Compute Module 3+ by thinkmassive on Monday January 28, 2019 @11:55AM (#19018987)
Oh yeah, I recommended these in the past and just mentioned them again in another comment on this thread :)

4GB is enough for most Pi projects, and <$20 premium is worth never needing to worry about SD reliability to me. My understanding is they use multi-level cells, which are far less expensive than true industrial grade single level cells, but they only write a single value to each cell, increasing the reliability significantly.

Re:New Raspberry Pi Compute Module 3+ by Fnoord on Monday January 28, 2019 @11:58AM (#19019007)
Industrial grade AMLC, for example from this reseller and brand, haven't let me down yet. Other HN users have a similar experience. The downside is the price. Delivery was one day from Springfield to The Netherlands but had to pay duties. Which is like a total opposite of China: bad quality, cheap, slow delivery, but no duties
Re:New Raspberry Pi Compute Module 3+ by StavrosK on Monday January 28, 2019 @11:10AM (#19018563)
Oof, pretty pricy, huh.

EDIT: I see in other comments that they're worth it, so they're a great option for when I need reliability at a premium, thanks!

Re:New Raspberry Pi Compute Module 3+ by jftuga on Monday January 28, 2019 @07:35PM (#19022620)
Check out: https://github.com/azlux/log2ram

Explanations: The script creates a /var/log mount point in RAM. So any writing of the log to the /var/log folder will not actually be written to disk (in this case to the sd card for a raspberry card) but directly to RAM. By default, every hour, the CRON will launch a synchronization of the RAM to the folder located on the physical disk. The script will also make this copy of RAM to disk in case of machine shutdown (but cannot do it in case of power failure). This way you avoid excessive writing on the SD card.

Re:New Raspberry Pi Compute Module 3+ by StavrosK on Monday January 28, 2019 @07:45PM (#19022672)
This is the best of both worlds, thanks!
Re:New Raspberry Pi Compute Module 3+ by thinkmassive on Monday January 28, 2019 @11:51AM (#19018946)
I had problems with Samsung and SanDisk cards, which worked fine in a GoPro, and troubleshooting the weird Pi symptoms was aggravating. So I tried aMLC cards from ATP and have had zero issues. They cost significantly more (~$20/ea for 4GB @ qty 10 from Digikey w/shipping) but it’s worth avoiding the hassle to me.

Obviously 4GB is not a lot, so I use tmpfs and USB storage as appropriate.

Re:New Raspberry Pi Compute Module 3+ by glenneroo on Monday January 28, 2019 @12:51PM (#19019444)
According to SanDisk support[0] they only support the High Endurance line of SD Cards[1] for Raspberry Pi.

[0]: https://www.raspberrypi.org/forums/viewtopic.php?p=1257129 (2nd to last comment in the thread)

[1]: https://www.amazon.de/gp/product/B00V5Q1K3O/

Re:New Raspberry Pi Compute Module 3+ by thinkmassive on Monday January 28, 2019 @03:27PM (#19020766)
Interesting, I definitely have not tried those yet. The Sandisk cards performed consistently worse in my experience, but I figured that could be anecdotal, and I had plenty of issues with Samsung as well.
Re:New Raspberry Pi Compute Module 3+ by leetbulb on Monday January 28, 2019 @11:08AM (#19018530)
Check out the NanoPI[0]. I've had really good luck with it.

Also, the Zymbit[1] works nicely with its eMMC.

0: http://nanopi.org/ 1: https://www.zymbit.com/zymkey/

Re:New Raspberry Pi Compute Module 3+ by StavrosK on Monday January 28, 2019 @11:12AM (#19018571)
That's interesting, thanks! Although, aren't AllWinner the devil? I think I recall someone saying that.
Re:New Raspberry Pi Compute Module 3+ by itdaniher on Monday January 28, 2019 @04:43PM (#19021379)
Maybe once upon a time, but with blobless mainline Linux kernel support for almost all features on almost all SOCs[1], including video codecs on many parts[2], anyone claiming Allwinner to be "the devil" or variants on a theme is unambiguously incorrect. Broadcomm and the raspi foundation are evil. [3][4]

[1] https://linux-sunxi.org/Linux_mainlining_effort

[2] https://www.kickstarter.com/projects/bootlin/allwinner-vpu-s...

[3] https://twitter.com/marcan42/status/1088472549715918848

[4] https://github.com/raspberrypi/firmware/blob/master/boot/LIC...

Re:New Raspberry Pi Compute Module 3+ by stefan_ on Monday January 28, 2019 @07:58AM (#19017082)
Of course, IO performance remains terrible.