https://shop.solid-run.com/product-category/embedded-compute...
You get upgradable ram, a PCIe slot for your favorite GPU/etc, and actual mainline linux support sufficient to boot most random linux distros (ok the onboard 10G+ nic might not work without patches).
Its not the fastest machine around (A72s) but there are 16 of them, so it does a decent job of building ARM software, and running VMs.
In fact I made the FreeBSD port of amdgpu work on my Macchiatobin :) Absolutely smooth experience btw, video output works even in UEFI (it actually runs the GOP driver from the card in QEMU), amdgpu works perfectly (played vkquake, supertuxkart, openmw, etc.)
[1]: https://www.jeffgeerling.com/blog/2020/external-gpus-and-ras...
So, yes you have to bring some storage, ram, ps and case. But one can cheap out and probably get all three for <$200, or go crazy and put 64G of ECC, and a few TB of storage in it. The cheaper off the shelf RAM/storage makes it a lot cheaper than a loaded mini with similar specifications.
And as another commenter points out, yes fairly common GPUs tend to work in it. Its not perfect, for that you have to spend more, but the time you will spend fighting with the mac mini+linux is going to be worth the difference.