If you go to huggingface.co, you can tell it what specs you have and when you go to a model, it'll show you what variations of that model are likely to run well.

So if you go to this[0] random model, on the right there is a list of quantifications based on bits, and those you can run will be shown in green.

[0] https://huggingface.co/unsloth/Mistral-Small-3.1-24B-Instruc...