30 - 10 years ago; Yocto (and its kin) served a very specific purpose. Building super tiny Linux distributions. If you have 4MiB worth of flash (storage) there's only so much you can cram in their.
Any "modern" linux distribution might still require more then that (excluding the likes of Linux from Scratch, as that falls in into the yocto category). Being CPU bound tends not to be the major issue, storage and RAM bound however is.
Personally, I've used debian in an embedded setting which can work great, but recently I've been using Alpine Linux for embedded and if you have a bit of storage (say 32 - 64MiB SPI NOR, or just some eMMC) you are good with Alpine as well.
Self-plug on not-yet-finished image builder around alpine: https://gitlab.com/esbs/builder
One other thing I think is greatly wrong with yocto, is the fact that it is super complex to do CI with it (because its one big thing) and build-times of 3.5 hours are ridiculous. Yes if I had to rebuild every single alpine package I'd probably end up with those build-times as well. But once a packages is built and vetted, just go with it (in your mirror if you will)
As for building the whole toolchain from scratch, sure, there might be exotic reasons to do that, but come on. Just stick your build environment in a container, and use that. Getting the toolchain from a "vendor" (alpine being one) is not a crime...
We ate the cost to upgrade the hardware simply so we could run mainstream Debian and punt Yocto to the garbage can.
If I'm being more charitable: Yocto has been made irrelevant by the progress of technology. Either I have a system that is capable of running full blown Linux or I have a system that has no hope of running anything that isn't completely bespoke. The number of times I have enough resources to run "I Can't Believe It's Not Linux" but not actual Linux has converged toward zero.