News for hackers. Stuff that matters.
Re:TinyStories: How Small Can Language Models Be and Still S... by raegis on Thursday January 02, 2025 @02:58PM (#42578152)
> Unfortunately, I have only seen 3 models, 3B or over, handle RAG.

What's the unit "B" in "3B"? I can search for acronyms like "RAG" just fine, but you experts aren't making it easy for us beginners :)

Edit: Apologies, this is obvious. My brain needed a reboot for the new year.

Re:TinyStories: How Small Can Language Models Be and Still S... by jmward01 on Thursday January 02, 2025 @03:06PM (#42578247)
(B)illion. It indicates the rough number of parameters in the model. Higher is generally more capable. 1B models are currently at the top end of 'easy' to deal with for playing around fine tuning and the like for most home lab setups.
Re:TinyStories: How Small Can Language Models Be and Still S... by cauliflower2718 on Thursday January 02, 2025 @03:04PM (#42578222)
You can ask an LLM exactly this question and it will tell you.

(The answer is billions of parameters)

Re:TinyStories: How Small Can Language Models Be and Still S... by SketchySeaBeast on Thursday January 02, 2025 @03:09PM (#42578282)
But what if they want to know they are finding the correct answer?
Re:TinyStories: How Small Can Language Models Be and Still S... by elliotto on Thursday January 02, 2025 @04:16PM (#42579008)
Asking anonymous people on a forum would be much better.
Re:TinyStories: How Small Can Language Models Be and Still S... by SketchySeaBeast on Thursday January 02, 2025 @05:00PM (#42579404)
At least a forum with domain-specific knowledge.
Re:TinyStories: How Small Can Language Models Be and Still S... by gpm on Thursday January 02, 2025 @05:07PM (#42579466)
And people to go "no, that's wrong" if someone posts something that's wrong.
Re:TinyStories: How Small Can Language Models Be and Still S... by greesil on Thursday January 02, 2025 @07:49PM (#42580867)
No, any answer will do.
Re:TinyStories: How Small Can Language Models Be and Still S... by cauliflower2718 on Friday January 03, 2025 @06:55AM (#42584850)
I think basic definitions for LLMs are solidly within the bounds of what we would expect e.g. chatgpt to be competent at. The task (defining terms) is simple and the specific content (basic LLM stuff) is easy to check by anyone who works on the LLM.

I agree with the general sentiment that we should not just blindly trust LLMs though.

Re:TinyStories: How Small Can Language Models Be and Still S... by jasonjmcghee on Thursday January 02, 2025 @03:10PM (#42578286)
tbf, the gp comment said 125MB and then 3B, which would be pretty confusing, as it's a typo and should be 125M.
Re:TinyStories: How Small Can Language Models Be and Still S... by jwineinger on Thursday January 02, 2025 @03:04PM (#42578231)
The number of parameters the model is trained on, in billions