Re:Infini-Gram: Scaling unbounded n-gram language models to ...
by mfornet on Sunday May 05, 2024 @06:35PM
(#40269216)
As I see it, this model will be able to predict “easy” to derive tokens but will no chance on “hard” tokens.
For example doing a sum of random numbers. If the token you are trying to predict is not in the training data, even if similar patterns exist, this model defaults to the Neural Model.
I guess then it is an aide to the neural model on filling the easy patterns.
For example doing a sum of random numbers. If the token you are trying to predict is not in the training data, even if similar patterns exist, this model defaults to the Neural Model.
I guess then it is an aide to the neural model on filling the easy patterns.