How Meta trains large language models at scale

Re:How Meta trains large language models at scale by burkaman on Thursday June 13, 2024 @09:31AM (#40669432)

It's actually a very common phrase on forums, I think because it's an actual error that Linux will report: https://askubuntu.com/questions/868321/gpu-has-fallen-off-th.... I've also never heard of it, but it seems like it must appear a lot in the training data and probably about 0 times is referring to a bus on the road.

[ Parent | on HN ]

Re:How Meta trains large language models at scale by TeMPOraL on Thursday June 13, 2024 @09:37AM (#40669491)

In my testing, both Llama 3 and its abliterated (uncensored) variant from[0] almost always remarked more or less directly that they see the joke in the phrase, so either they've seen the other meaning in training, or inferred it.

[0] - https://news.ycombinator.com/item?id=40665721

[ Parent | on HN ]

Re:How Meta trains large language models at scale by burkaman on Thursday June 13, 2024 @10:05AM (#40669815)

Oh I agree it probably inferred the joke. I was actually more surprised that it knew the real meaning of the phrase because I as a human did not, until I looked it up and saw how common it is.

[ Parent | on HN ]

Re:How Meta trains large language models at scale by Technetium on Thursday June 13, 2024 @10:46AM (#40670370)

Please use the word ablated instead. That article's title is not using a real word. I'm assuming it's the author's English issue, since they called the model "helpfull" instead of "helpful".

[ Parent | on HN ]

Re:How Meta trains large language models at scale by TeMPOraL on Thursday June 13, 2024 @11:39AM (#40671008)

Oops. I actually originally wrote "ablated", then changed it to be consistent with the title.

[ Parent | on HN ]