33TB of text data for a 1T-parameter model

Re:33TB of text data for a 1T-parameter model by visarga on Sunday April 16, 2023 @12:23AM (#35587251)

They might fear unwanted memorsation/regurgitation of training content.

[ Parent | on HN ]

Re:33TB of text data for a 1T-parameter model by ALittleLight on Sunday April 16, 2023 @03:23AM (#35587996)

But shouldn't they try it? Has someone tried it?

[ Parent | on HN ]

Re:33TB of text data for a 1T-parameter model by thomashop on Sunday April 16, 2023 @03:57AM (#35588108)

This is already how it works. When you train a deep learning model you show the samples of the training data hundreds of times. Of course depends on the size of your training set and your budget, etc.

You often train for 100s of epochs. One epooch means one pass over the training set.

[ Parent | on HN ]