There's also the DSP subset where people need to write inner loops in highly platform-specific assembler. e.g. my employer Cirrus has our own "Coyote" architecture for which we have a C compiler: https://statics.cirrus.com/pubs/proDatasheet/CS4970x4_F1.pdf
"Cheap" isn't just about BOM price, the power consumption matters too.
We already had plenty of high level languages running on CP/M, MS-DOS, Amiga, Atari, ... class hardware.
Those languages would already fly on ESP32.
I know I can do a wrist watch with a Raspberry Pi, but will it be profitable/convenient?
So basically an Intel Pentium from 1994, so from 26 (!) years ago. Size: 90 mm2, maximum power consumption: 10W, price at launch: $699.
Buuuut: production process 600nm.
Today every Intel CPU uses a 14nm production process, but let's go for a cheaper option and "use" the 22nm.
So the gate size is ~30x smaller today. And I know that the relation between the gate size and the die size is not linear (2x smaller gate size usually leads to a die size which is smaller by more than 2x), but let's go with the 30x, to keep things simple.
For the power I have no idea, but I'd assume that the power consumption would go down at least 30x.
Price: hard to say, but considering how hardware evolves, definitely more than 30x cheaper :-)
So a Intel Pentium with modern technology could look something like this:
Size: 3mm2 (probably less), maximum power consumption: 0.3W (probably way less), price at launch: $30 (I'd argue that it would be closer to 0.3$ :-) ).
I could be way off in the weeds here...
My guess is that it's just a matter of existing tooling, expertise, human resistance to change, companies wanting to a) not risk anything and b) to nickel and dime everything.
I'm arguing for an x86 because that way you could throw any kind of modern tooling at it. ARM or MIPS would probably be better candidates.
You can use a small ARM with integrated flash, RAM and LCD controller. Does it need MMU for Linux? There is family of products for that. Or use another family for bare-metal systems.
You can see how the thing starts to complicate regarding architectures/platforms.
But my main point was to dump super old and limited architectures when these days we can economically use modern architectures we use everywhere else (x86/ARM/MIPS, whatever). If we wanted to, we could literally hoist designs from 20+ years ago and use newer production technologies to make them embeddable.
Using mainstream tech stacks is very empowering. Updated compilers everywhere, many programming languages and stacks with huge communities everywhere, modern debugging tools at low/no cost, fast debug cycles, etc.
But there's little interest in this because which self-respecting hardware maker would commoditize its own products? :-D
The problem is not the processor, it's the lack of MMU and/or special DMA or interrupt engines and special GPIO. No kernel is potentially ported to such custom infrastructure, and if you have a very tiny flash, few of the ones which could will fit. (Say, target L4, porting Fiasco or OKL4.)
We've had 32 bit x86 CPUs since 1985, surely we could produce a 100Mhz one for cheap enough that nobody would need to use 8 bit ones in 2020? I know that I'm just daydreaming and companies are stingy...