News for hackers. Stuff that matters.
Re:NYT: The Data That Powers AI Is Disappearing Fast by skybrian on Monday July 22, 2024 @01:51AM (#41031262)
There are probably a lot of applications where the LLM could rely more on data that's supplied to it just-in-time in the context window, and less on specialist knowledge from its training set.

Also, "natural" data taken from the Internet is probably quite inefficient as training material. It's going to have a lot of duplication. You only need each fact once to be able to synthesize more examples of it.

There are no comments on this comment yet.