News for hackers. Stuff that matters.
Re:Visual ChatGPT by muyuu on Friday March 10, 2023 @06:06AM (#35092953)
if the intuition is based on images taking more space to encode than text, then it's a false intuition because the size of models is not correlated to the encoding size of the individual data points - it's correlated instead to the amount of relevant choices the model can make and the complexity of the dynamics in the model

I don't think general assertions like "language is more complicated" are congruent or meaningful, it really depends on what the model is trying to achieve; it's the complexity of that which will require a larger or smaller model

There are no comments on this comment yet.