Re:I disagree with Geoff Hinton regarding "glorified autocom...
by Guvante on Wednesday November 22, 2023 @02:50AM
(#38376235)
Except that if you narrow to a tiny training set you are back to problems that can be solved almost as quickly with full text search...
Re:I disagree with Geoff Hinton regarding "glorified autocom...
by xcv123 on Wednesday November 22, 2023 @04:50AM
(#38377159)
Narrow to a tiny training set? What are you talking about now? That has nothing to do with deep learning.
GPT-3.5 was trained on at least 300 billion tokens. It has 96 layers in its neural network of 175 billion parameters. Each one of those 96 stacked layers has an attention mechanism that recomputes an attention score for every token in the context window, for each new token generated in sequence. GPT-4 is much bigger than that. The scale and complexity of these models is beyond comprehension. We're talking about LLMs, not SLMs.
Sloppy non-rigorous shallow arguments are bad for discussion.