Re:Qwen2 LLM Released
by algo_trader on Thursday June 06, 2024 @02:00PM
(#40600451)
what would you recommend as a maximum-2B model for coding/reasoning domain ?
[ Parent
| on HN ]
For reasoning tasks and coding tasks where you’re chatting with the model, there are no 2B models that I would recommend at this point.
> maximum-2B model
If you are resource limited, remember that you can also play with the quantization to fit more parameters into less amount of RAM. Phi-3-mini [1] (a 3.8B model) is 7.64GB with full (16-bit floating point) precision, but it is only 2.39GB when quantized to 4 bits.
That being said, I haven't personally tested it, but have heard good things for CodeGemma 2B [2].
[1] https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf...
CodeGemma-2b does not come in the "-it" (instruction tuned) variant, so it can't be used in a chat context. It is just a base model designed for tab completion of code in an editor, which I agree it is pretty good at.