News for hackers. Stuff that matters.
Re:Why Meta’s latest large language model survived only thre... by j-krieger on Friday November 18, 2022 @04:26PM (#33662486)
Stories like these are the reason why I immediately download and backup software if I even get a tiny inkling that it may be „too free“.
Re:Why Meta’s latest large language model survived only thre... by Philadelphia on Friday November 18, 2022 @05:12PM (#33663124)
“A fundamental problem with Galactica is that it is not able to distinguish truth from falsehood, a basic requirement for a language model designed to generate scientific text. People found that it made up fake papers (sometimes attributing them to real authors), and generated wiki articles about the history of bears in space as readily as ones about protein complexes and the speed of light.”

How would anyone think releasing this was a good idea?

Re:Why Meta’s latest large language model survived only thre... by vba616 on Friday November 18, 2022 @06:42PM (#33664582)
People have been expressing surprise about this issue with computing machinery for 200 years.
Re:Why Meta’s latest large language model survived only thre... by mfld on Saturday November 19, 2022 @03:51AM (#33668377)
1. There is a large corpus of scientific publications.

2. Thanks to scihub et al, it is possible to download them en masse.

3. It is now feasible to train good language models on a large text corpus.

4. Wouldn't it be interesting to see what a model trained on this data comes up with? Maybe even new connections in existing knowledge?

Re:Why Meta’s latest large language model survived only thre... by neweroldguy on Saturday November 19, 2022 @04:40AM (#33668623)
> How would anyone think releasing this was a good idea?

Why would it not be? A model built on any consistent large data set could be interesting I’d expect.

But just because the data set has some semantic property, that doesn’t mean that a model built from it will necessarily generate text that had the same property (in this case, “being scientifically correct”). The real question is why would anyone think that?

Re:Why Meta’s latest large language model survived only thre... by p0pcult on Saturday November 19, 2022 @10:09AM (#33670771)
Good points on both sides. I think the answer to your second question is, "first mover advantage, even if the product is bad"
Re:Why Meta’s latest large language model survived only thre... by bpanon on Wednesday November 23, 2022 @07:08PM (#33726200)
Humans do the dame thing.