News for hackers. Stuff that matters.
Re:MiniGPT-4 by sigmoid10 on Tuesday April 18, 2023 @09:13AM (#35613543)
The UAT states that depth is fundamentally not important, at least theoretically. It only has immense practical uses. So adding an intermediate linear layer + some nonlinearity already gets you an error scaling like O(1/N) for width N (in theory), regardless of what you are actually mapping. At least as long as it's somewhat continuous.

There are no comments on this comment yet.