marknsikora

Most of these older cards are lacking the physical hardware for fp8 or other lower precisions that most quantized models use. Or the memory to run models at higher precision.

Yea. When people talk about how simple things were in the past, I think they oblivious to the fact we were all writing out X11 configs by hand and having them break in all sort of strange and interesting ways. Open source has always been very "I gave it to you for free, I'm not going to hold your hand".

You hang out on weird hardware hacker forums and spot what people are working on. You can try buying the things multiple people are working on and getting good early results on. But most of it will end up being ewaste.

Good article. But the Makefile part should really be replaced by tox as a best practice. Tox even has a uv runner now that will handle setting up all the environments.