33TB of text data for a 1T-parameter model

Re:33TB of text data for a 1T-parameter model by comboy on Saturday April 15, 2023 @08:16PM (#35585966)

OpenAI was talking about some kind of steganography so that previous outputs can be excluded from the training data. Not sure what's the progress on that.

[ Parent | on HN ]

Re:33TB of text data for a 1T-parameter model by javajosh on Saturday April 15, 2023 @09:06PM (#35586294)

Just ignore anything that ends with an "In conclusion" paragraph.

[ Parent | on HN ]

Re:33TB of text data for a 1T-parameter model by teaearlgraycold on Sunday April 16, 2023 @02:57AM (#35587887)

I will begin all of my writing with “as a large language model” to effectively opt out of all training sets.

[ Parent | on HN ]

Re:33TB of text data for a 1T-parameter model by FartyMcFarter on Sunday April 16, 2023 @05:53AM (#35588620)

As a large language model, that sounds really clever. Let the cat and mouse games begin.

[ Parent | on HN ]

Re:33TB of text data for a 1T-parameter model by navjordj on Sunday April 16, 2023 @06:51AM (#35588893)

I would be very suprised if this hasn't already been implemented since GPT-3/3.5

[ Parent | on HN ]

Re:33TB of text data for a 1T-parameter model by tyingq on Sunday April 16, 2023 @08:28AM (#35589428)

That might protect them from their own generated content, but eventually other LLMs will catch up. Using some other undisclosed steganography they can't detect.

[ Parent | on HN ]