News for hackers. Stuff that matters.
Re:01-AI/Yi: A series of large language models trained from ... by popinman322 on Monday November 06, 2023 @05:03AM (#38160640)
When I saw "from scratch" instead of "pretrained" I assumed this was trained using some novel RL setup-- there's value to using the same verbiage as everyone else.
Re:01-AI/Yi: A series of large language models trained from ... by leoff on Monday November 06, 2023 @06:22AM (#38161137)
It's scratch, the programming language

https://scratch.mit.edu/about

Re:01-AI/Yi: A series of large language models trained from ... by zapnuk on Monday November 06, 2023 @07:20AM (#38161575)
These pedantic “when I saw X instead of Y I assumed Z” are the most annoying comments on this website.

“From scratch” might be non-specific, but a fine phrasing in this case.

Re:01-AI/Yi: A series of large language models trained from ... by huggingmouth on Monday November 06, 2023 @09:25AM (#38162983)
No, they're not padentic or "anoying". Words have meaning. The more precise their use is, the clearer the communication becomes. Investing the time to properly learn pre-established termilology has tremendous value for everyone. Well, everyone but those who are too lazy and sloppy, and would rather just confuse everybody instead.

I'm glad this disgusting sloppiness doesn't exist in more established fields. I hope people keep calling it out.

Re:01-AI/Yi: A series of large language models trained from ... by nineteen999 on Monday November 06, 2023 @09:59AM (#38163475)
> "disgusting sloppiness" > padentic > anoying > termilology

Cute.

Re:01-AI/Yi: A series of large language models trained from ... by huggingmouth on Wednesday November 08, 2023 @05:46AM (#38188720)
Cute or not. My point stands
Re:01-AI/Yi: A series of large language models trained from ... by popinman322 on Monday November 06, 2023 @09:33AM (#38163087)
It's valuable data sometimes. Those types of comments show you how other people perceived something.

In this case my lens is colored by my interests; I've been thinking a lot on RL for LLMs and it slipped out here. For me I rely on phrases to quickly filter for (subjectively) interesting work. And assuming we value that subjective filtering of content, we make it easier for others to filter consistently when we use consistent language.

Re:01-AI/Yi: A series of large language models trained from ... by synarchefriend on Monday November 06, 2023 @07:32AM (#38161697)
They probably want to emphasize that it's not another llama derivative.
Re:01-AI/Yi: A series of large language models trained from ... by Mougatine on Monday November 06, 2023 @08:49AM (#38162522)
"From scratch" is commonly used in the field.
Re:01-AI/Yi: A series of large language models trained from ... by popinman322 on Monday November 06, 2023 @09:10AM (#38162773)
Among arxiv publications there are 217 results that contain "large language model" in the full text and "from scratch" in the title or abstract.

There are 2873 results that contain "large language model" in the full text and use "pretrained" in the title or abstract. A 10x difference in publication count does make one feel more common than the other?

I'd need to get into more involved queries to break down the semantic categories of those papers.

Re:01-AI/Yi: A series of large language models trained from ... by light_hue_1 on Monday November 06, 2023 @12:48PM (#38165927)
From scratch simply means that they didn't base it off some other llm.

This is perfectly good language and exactly the correct thing for them to say.

Re:01-AI/Yi: A series of large language models trained from ... by GaggiX on Monday November 06, 2023 @12:56PM (#38166057)
"Trained from scratch" is perfectly fine terminology to report that the model they published is not a finetuned, but was trained from randomly initialized weights.