Re:NYT: The Data That Powers AI Is Disappearing Fast
by skeledrew on Sunday July 21, 2024 @08:32PM
(#41029758)
No need even for synthetic data. Advance away from doing statistics on troves of human-generated data, to doing statistics on raw environmental data, like humans do. This will also likely mean moving away from the GPT architecture completely.
[ Parent
| on HN ]
Re:NYT: The Data That Powers AI Is Disappearing Fast
by Kostchei on Sunday July 21, 2024 @09:06PM
(#41029960)
This is the way. Robotics, self driving cars, mobile devices themselves, either phone or glasses....
They can generate the data needed for the next steps. (pun intentional)
Text was easy, the process has been defined. Multi-modality improves the models. The next set of data is the world. Everything we currently observe using our senses and data outside our senses like radio waves. Build a better world model.
Re:NYT: The Data That Powers AI Is Disappearing Fast
by FeepingCreature on Sunday July 21, 2024 @08:43PM
(#41029831)
Seems silly to say: "Instead of training this LLM on internet data, we will train the LLM on the output of an OCR model pointed at a screen scrolling internet data. This is clearly more ethical."
Re:NYT: The Data That Powers AI Is Disappearing Fast
by skeledrew on Sunday July 21, 2024 @09:23PM
(#41030041)
I made no allusion to ethics here; merely logic. Humans don't learn by scraping the web, but by observing the environment. The web is an incomplete distillation of humanity's knowledge of facts and various interpretations of them (which also includes "creative" works as one cannot create anything without basing said creations on environmental observations).