Re:XGen-7B, a new 7B foundational model trained on up to 8K ...
by MINIMAN10000 on Wednesday June 28, 2023 @10:56PM
(#36515793)
You would just need a computer which can fit 2 3090s in order to run those to run something like TheBloke/airoboros-65B-gpt4-1.3-GPTQ
https://www.reddit.com/r/LocalLLaMA/wiki/models/ gives you a list of VRAM requirements to load the model into GPU VRAM. the more VRAM the computer has, the larger the model you can load in, thus making 3090s the current consumer grade king due to price to max VRAM.
This being said however most models are LLAMA based which all fall under that specific research license.
So following the rules, you would be limited to a subset of models which are foundational models which allow for commercial use
https://www.reddit.com/r/LocalLLaMA/wiki/models/ gives you a list of VRAM requirements to load the model into GPU VRAM. the more VRAM the computer has, the larger the model you can load in, thus making 3090s the current consumer grade king due to price to max VRAM.
This being said however most models are LLAMA based which all fall under that specific research license.
So following the rules, you would be limited to a subset of models which are foundational models which allow for commercial use