Re:Google Open-Sources Trillion-Parameter AI Language Model ...
by ma2rten on Wednesday February 17, 2021 @09:50PM
(#26174886)
Yes, it has been shown [1] that these models can memorize information even if it only appears once in the training data. It could potentially cause issues with privacy and especially copyright. However, in this case it's not relevant because they didn't release the model weights.
[1] https://arxiv.org/abs/2012.07805