OPT: Open Pre-trained Transformer Language Models

Re:OPT: Open Pre-trained Transformer Language Models by tsol on Tuesday May 03, 2022 @01:36AM (#31245020)

Are there really no moderated forums that the data can be taken from? Even HN-based training data would be much more civil

[ Parent | on HN ]

Re:OPT: Open Pre-trained Transformer Language Models by Gigachad on Tuesday May 03, 2022 @01:55AM (#31245123)

A model trained on HN would spit out a 5 paragraph story about how minorities provide a negative ROI for cities. Or how the homeless need to removed from society.

[ Parent | on HN ]

Re:OPT: Open Pre-trained Transformer Language Models by can16358p on Tuesday May 03, 2022 @02:56AM (#31245479)

Don't forget that it must also generate, at some point regardless of the topic, a new terminal emulator, and an extremely positive or extremely negative opinion about how blockchain can solve a problem.

[ Parent | on HN ]

Re:OPT: Open Pre-trained Transformer Language Models by IAmEveryone on Tuesday May 03, 2022 @02:52AM (#31245458)

Sure, but it would never do something actually bad, like raising the possibility that sexual harassment might, sometimes, be an issue, or questioning the value of phrenology.

[ Parent | on HN ]

Re:OPT: Open Pre-trained Transformer Language Models by sanxiyn on Tuesday May 03, 2022 @01:41AM (#31245039)

Note that HN is included in the training data, see page 20.

[ Parent | on HN ]

Re:OPT: Open Pre-trained Transformer Language Models by IAmEveryone on Tuesday May 03, 2022 @02:53AM (#31245464)

Go figure (8)!

[ Parent | on HN ]