News for hackers. Stuff that matters.

A comment on Qwen2 LLM Released. Parent comment

Re:Qwen2 LLM Released by minimaxir on Thursday June 06, 2024 @12:59PM (#40599724)
A properly finetuned model can perform better for a use case but even with PEFT/LoRAs, finetuning and managing "smaller" open-source LLMs (7B params) models like Llama3 is annoying. That's partially why the even-smaller ~2B Phi series of models took off.

A 0.5B model may not be that great out of the box but there's a lot of oppertunity if it's responsive to finetuning.

[ Parent | on HN ]
Re:Qwen2 LLM Released by danielhanchen on Friday June 07, 2024 @01:37AM (#40605627)
Ye smaller models are fantastic for finetuning and probably on device applications - they can act as like a first pass on most LLM applications, then if it requires a larger model to intervene, then it can pass if off to some larger one. I do have a Colab to finetune 0.5b 2x faster here for those interested: https://colab.research.google.com/drive/1-7tjDdMAyeCueyLAwv6...
[ Parent | on HN ]