Re:Google CEO says more than a quarter of the company's new ...
by wahnfrieden on Thursday October 31, 2024 @12:38PM
(#42008548)
- context of millions of tokens is frontier
- context over training is like someone referencing docs vs vaguely recalling from decayed memory
- context caching
Re:Google CEO says more than a quarter of the company's new ...
by layer8 on Thursday October 31, 2024 @01:35PM
(#42009197)
You’re assuming that everything can be easily known from documentation. That’s far from the truth. A lot of what LLMs produce is informed by having been trained on large amounts of source code and large amounts of discussions where people have shared their knowledge from experience, which you can’t get from the documentation.
Lastly, it’s also a huge waste of energy to feed the same information over and over again for each query.