A small model, yes! But not necessarily a good model.

With the additional caveat that I don't know whether that specific card is supported by modern drivers.

You'd be looking at one in the 6B or 7B parameters range at FP8. Or smaller. It's been quite some time since a recognizable company in the AI space released a model that small. You can try larger model that has been quantized down to that size, but they don't always fare well with that.

Modern text-to-speech and speech-to-text models also fit well into modest amounts of VRAM.

You're arguing for a very specific range of weights but many slightly smaller and slightly larger models have been released including QAT and MoE versions.

An old nVidia brand card with 8GB is more than enough to see those models running at usable speeds and accuracy.

I wasn't arguing for or against anything, just trying to help answer the person's question.

I'm able to run qwen 3.6 35b a3b with my rtx 2060 super which also has 8gb of vram. It's a very capable model, not as smart as a frontier model but it is still very useful.

I use an 6bit quant and get around 20 tokens per second.

Sure. The fastest small coding model is probably Mellum2 12B-A2.5 by Jetbrains. It matches or beats all Qwen models in this class.

Can even run on a notebook CPU and comes in Base (best for FIM), Instruct and Thinking variants. mradermacher has imatrix quants for people who can't run it at Q8.

IQ4 should fit, but even if it doesn't, llama.cpp has options to partially offload models to system memory.