That's not true, systemd has an explicitly documented and supported initrd interface: https://systemd.io/INITRD_INTERFACE/

It's actually really easy, there's rarely a reason for the initrd to be more complex than a single 50-line busybox script on an embedded device with built-in drivers.

Dracut is used both in Void Linux and on Alpine without systemd and with busybox.

It even runs continuous integration with musl based containers.

It has an init:

  $ /home/busybox-1.35 init --help
  BusyBox v1.35.0 (2022-01-17 19:57:02 CET) multi-call binary.

  Usage: init

  Init is the first process started during boot. It never exits.
  It (re)spawns children according to /etc/inittab.
  Signals:
  HUP: reload /etc/inittab
  TSTP: stop respawning until CONT
  QUIT: re-exec another init
  USR1/TERM/USR2/INT: run halt/reboot/poweroff/Ctrl-Alt-Del script

That is almost by definition not an embedded device. There's a reason we have vfork().

Been a while since I was in the digital signage space but a lot of the equipment runs of the shelf RK3288 plugged into commercial displays. 2GB of RAM was pretty common. IIRC's though LG's WebOS TV's have minimum 2GB of RAM in the digital signage space directly built into the units themselves. I believe Samsung Tizen based units has similar RAM.

My router has 1GB of RAM in it. But even my cheapest routers have 128 to 256 MB of RAM. The Cisco 9300 Catalyst switches have about 8 GB of RAM, but switches with beefy amounts of RAM are getting pretty common now, even if somewhat pricey.

Yeah there's massive swathes of embedded space that's tiny. But the higher end stuff isn't exactly out of reach anymore. The RK3288's IIRC ran about $20 a unit at the time before I left the industry.

In 2014 i got myself a second- or third-hand thinkpad X220, released in 2011, off eBay. It came with 8gb ram (two 4GB sticks) but it supported 16GB ram as well (two 8gb sticks).

The laptop (Asus A8Jc) I got when I was a teenager in ~2005 came with a dual core intel cpu and 1GB ram. So "512mb desktops" are way older than that.

If they can't afford those specs then you're approaching the group of people who can't afford a computer in the first place.

I recently wanted an MP3 player for an art project. Local stores don't sell mp3 players anymore, they only sell smartphones. So I bought an MVNO smartphone for $40. When I charged it up and tried to use it, I thought maybe it was broken, because it would take 10-30 seconds to load a settings menu or app. Nope, all these bargain carrier-branded phones are that slow. The hardware is [somewhat] old, but the new Android OSes run like molasses on them. It was like going back in time. Remember how Windows 98 would make your hard drive screech for a good couple minutes as it struggled to juggle the swap memory so you could open MS Word? That's the experience with most software today with "affordable" hardware even a few years old.

So using Windows XP is often the only choice, if you don't have a lot of money, like 1/3rd of the planet. (And it's not just the third world. 59% of American households with K-12 school kids don't have a working computer, or it works too slowly to be useful)

So you bought a new phone for $40, and it was a POS?

My kids use my old iPhone 7, which is in the same price bracket and is nothing like that. Its fast enough for Roblox, Minecraft, and certainly fast enough for a web browser.

I have an old Dell USFF that I use for server purposes, but its a Skylake (so newer than what was the original conversation), with ssd and 16Gb, and that was <£50. That can boot with systemd in under 5 seconds. It can boot to the full Gnome desktop in under 6 seconds. Firefox can start and get the Office.com site up in less than 3 seconds.

Because thats what were talking about.

> But desktops with those specs are <$100, probably less than $50.

I just checked eBay. Yes they still are.

That’s not typical at all. I’ve done a lot of work with production runs in the millions. There is no way we’d put that much RAM on a device unless it was absolutely, undeniably required for core functionality.

That doesn't pass the sniff test. Look at retail RAM prices. Certainly the magnitude of the price is quite different than buying individual RAM chips at quantity, but the costs do scale up as RAM size goes up. Hell, look at RAM chip prices: you are definitely going to increase the price by more than a negligible amount if you 2x or 3x the amount of RAM in your design.

Also consider the Raspberry Pi, since the article mentions it quite a bit: RAM size on the Pi is the sole driver of the different price points for each Pi generation.

So no, at those sizes price really does not change all that much, and installing 512MB of RAM instead of 64MB only increases the product's cost by $0.15. It's a commodity made on legacy processes, things like packaging, testing, and handling far outweigh the cost of the actual transistors inside the chip.

[0]: https://www.lcsc.com/product-detail/DDR-SDRAM_Winbond-Elec-W...

[1]: https://www.lcsc.com/product-detail/DDR-SDRAM_Winbond-Elec-W...

[2]: https://www.lcsc.com/product-detail/DDR-SDRAM_Samsung-K4B2G1...

[3]: https://www.lcsc.com/product-detail/DDR-SDRAM_Samsung-K4B4G1...

[4]: https://www.lcsc.com/product-detail/DDR-SDRAM_Samsung-K4A8G1...

I remember arguing as well with people about 0.10$ components. They told me it was a no go, not even a point of bringing it up. Sometimes even a 0.01$ is a big deal.

There are a lot of other factors involved with respinning hardware which make an upgrade a lot more expensive than simply a BOM increase. I can definitely understand why an existing product wouldn't be upgraded, but for a new product going to a bigger memory chip is a far smaller hurdle. The added software engineering time for working around RAM limitations can easily outweigh any money saved on the BOM, with choosing a smaller chip ending up being penny-wise pound-foolish.

And indeed, an extra $0.10 or even $0.01 can be a big deal. But those cheap systems usually aren't powerful enough to meaningfully run Linux in the first place: just because you can technically hack a $1.00 RP2040 or ESP32 into running Linux doesn't mean it is a good idea to actually do so. If your product is both cheap enough that it represents a significant fraction of your BOM and high-volume enough that you can afford the engineering time in the first place, why not use a purpose-built embedded OS like Zephyr?

In my experience production people will eat your soul for a single resistor if they can cut costs on it.

Also, just because something holds true at large numbers doesn't mean it scales all the way down. Either due to economies of scale, or the negligibly different architecture/components at that size.

In Automotive (i.e. telematics devices) you'll want a separate MCU for CAN-bus. For example, if you are doing Request-Response model you'll want to make use of the built-in filters. Besides, it is unlikely that a modem would support the CAN interface.

In Cellular IoT you'll prefer a separate MCU as it is much easier to port to different hardware. For example, you can hook-up the module via UART and use CMUX (AT/GNSS/PPP) and you'll cover 80%+ of the modules available in the market with very minimal specific implementation layers to enter into these modes.

DESCRIPTION vfork() was originally used to create new processes without fully copying the address space of the old process, which is horrendously inefficient in a paged environment. It was useful when the purpose of fork(2) would have been to create a new system context for an execve(2). Since fork(2) is now efficient, even in the above case, the need for vfork() has diminished. vfork() differs from fork(2) in that the parent is suspended until the child makes a call to execve(2) or an exit (either by a call to _exit(2) or abnormally). ... HISTORY The vfork() function call appeared in 3.0BSD with the additional semantics that the child process ran in the memory of the parent until it called execve(2) or exited. That sharing of memory was removed in 4.4BSD,

The gap between “over 1GB of RAM” and 8MB RSS contains the vast majority of embedded Linux devices.

I, too, enjoy when the RAM budget is over 1GB. The majority of cost constrained products don’t allow that, though.

That’s said, it’s more than just RAM. It increases boot times (mentioned in the article) which is a pretty big deal on certain consumer products that aren’t always powered on. The article makes some good points that you’ve waved away because you’ve been working on a different category of devices.

The gap between 16MB RAM and 64MB RAM doesn't exist, though. Literally doesn't, the components have the same cost down to a cent in the BOM.

And if you can have 64MB, then there's systemd's own true memory use (around 3-4MB) is completely immaterial.

just look at wifi routers. in the usa and China they are all sold with 64 or 128mb ram. south America and Europe they are all 16 or 32 for no clear reason.

My ISP provided router also does VPN, VoIP, mesh networking, firewalling, and it's towards the lower end of feature set (as it's offered for free and not a fancy router I bought).

Are you talking about devices from the early 2000?

pick any modem from linksys or dlink or netgear. then buy one in south America and compare what's really inside

look at all the revB on openwrt wiki, sometimes ram lowers. sometimes arm cpu change to mediatek. often the wifi chip changes from qualcomm to rtl. and it's always the revisions sold outside of usa and China in the observation fields.

I don't know where you're getting your data from but it's clearly wrong or outdated. These are the most often sold routers in Czechia on Alza (the largest online retailer) under $100:

- TP-Link Archer AX53 (256MB)

- TP-Link Archer AX23 (128MB)

- TP-Link Archer C6 V3.2 (128MB)

- TP-Link Archer AX55 Pro (512MB?)

...

- Mercusys MR80X (256MB)

- ASUS RT-AX52 (256MB)

https://www.alza.cz/EN/best-sellers-best-wifi-routers/188430...

Of all currently existing Linux devices running around the world right this moment? Maybe.

But of new devices? Absolutely not, and that's what I'm talking about.

> The majority of cost constrained products don’t allow that, though.

They increasingly do allow for it, is the point I'm trying to make.

And when they don't: there are far better non-Linux open source options now than there used to be, which are by design better suited to running in constrained environments than a full blown Linux userland ever can be.

> It increases boot times (mentioned in the article) which is a pretty big deal on certain consumer products that aren’t always powered on. The article makes some good points that you’ve waved away because you’ve been working on a different category of devices.

I've absolutely worked on that category of devices, I almost never run Linux on them because there's usually an easier and better way. Especially where half a second of boot time is important.

The trouble with "new" is that it keeps getting old.

There would have been a time when people would have said that 32MB is a crazy high amount of memory -- enough to run Windows NT with an entire GUI! But as the saying goes, "what Andy giveth, Bill taketh away". Only these days the role of Windows is being played by systemd.

By the time the >1GB systems make it into the low end of the embedded market, the systemd requirements will presumably have increased even more.

> there are far better non-Linux open source options now than there used to be, which are by design better suited to running in constrained environments than a full blown Linux userland ever can be.

This seems like assuming the conclusion. The thing people are complaining about is that they want Linux to be good in those environments too.

Those days are long gone though, for better or worse.

We live in the 2020s now and ram is plenty. The small computers we all carry in our pockets (phones) usually have between 4 and 16g GB ram.

I do not think the monster CPUs running Android or iOS nowadays are representative of embedded CPUs.

RAM still requires power to retain its contents. In devices that sleep most of the time, decreasing the amount of RAM can be the easiest way to increase battery life.

I would also think many of the small computers inside my phone have less memory. For example, there probably is at least one CPU inside the phone module, a CPU doing write leveling running inside flash memory modules, a CPU managing the battery, a CPU in the fingerprint reader, etc.

And the progress has kind of stalled:

https://aiimpacts.org/trends-in-dram-price-per-gigabyte/

We've been stuck at ~$10/GB for a decade. There are plenty of devices for which $10 is a significant fraction of the BOM and they're not going to use a GB of RAM if they can get away with less. And if the hardware price isn't giving you a free ride anymore, not only do you have to stop the software from getting even bigger, if you want it to fit in those devices you actually need it to get smaller.

You can say the free lunch is still there, but it's gone from a buffet to a stick of celery.

Is it really the case? On desktops it is significantly faster than all the other alternatives. Of course if you do know your hardware there is no need for discovering stuff and the like, but I don't know. Would be interested in real-life experiences because to me systemd's boot time was always way faster than supposedly simpler alternatives.

Later, when I got myself a laptop with an SSD, I discovered that what my older Arch configuration could do on an HDD is what systemd could do only with an SSD.

but the "faster boot" you're remembering are actually a joke at the time. since the team working on it were probably booting vms all the time, the system was incredible aggressive on shutdown and that was the source of it. something like it reboots so fast because it just throws everything out and reboot, or something. i don't really care much for the jokes but that was why everyone today remembers "systemd is fast".

Nowadays if you start anything more serious from your user session (e.g. start a qemu vm from your user shell) it will get SIGHUP asap on shutdown, because systemd doesn't care about non service pids. but oh well.

...which is where the jokes about "systemd is good for really fast reboots" came from mostly.

It's trivial to run these as a user service, which can linger afterwards. Also, systemd has a configurable wait time before it kills a process (the "dreaded" 2 mins timer is usually something similar)

I'm not sure what you mean by "properly" but didn't initng and upstart (and probably some others I can't recall) do the parallel stuff before systemd?

Systemd already parallelises by default so I don't know what insanely strange things you were doing but I fail to see how it could bring boot time form 11s to 1 minute. Also, it's very easy to get a list of every services enabled with systemctl (systemctl list-unit-files --state=enabled) so I don't really know what your point about a giant graph is.

If you're working in the embedded space it's surely worth a little bit of time to optimize something like this.

All the smaller systems with no RAM run on baremetal anyway. There's no room and no need to run Linux or a threaded RTOS. Much less security headaches also.