News for hackers. Stuff that matters.

A comment on OpenAI cofounder Ilya Sutskever says the way AI is built is about to change. Parent comment

Re:OpenAI cofounder Ilya Sutskever says the way AI is built ... by exe34 on Saturday December 14, 2024 @02:39AM (#42415360)
> Humans doesn't need trillions of tokens

I wonder about that. we can fine tune on calculus with much fewer tokens, but I'd be interested in some calculations of how many tokens evolution provides us (it's not about the DNA itself, but all the other things that were explored and discarded and are now out of reach) - but also the sheer amount of physics learnt by a baby by crawling around and putting everything in its mouth.

[ Parent | on HN ]
Re:OpenAI cofounder Ilya Sutskever says the way AI is built ... by YetAnotherNick on Saturday December 14, 2024 @02:52AM (#42415393)
Yes, as I said in the last comment. With current training techniques, one internet data is enough to give models what is given by evolution. For further training, I believe we would need different techniques to make the model self aware about its knowledge.

Also, I believe a person who is blind and paralyzed for life could still attain knowledge if educated well enough.(can't find any study here tbh)

[ Parent | on HN ]
Re:OpenAI cofounder Ilya Sutskever says the way AI is built ... by exe34 on Saturday December 14, 2024 @03:06AM (#42415444)
yeah blind and paralysed from birth - I'm doubtful that hearing along would give you the physics training. although if it can be done, then it means the evolutionary pre-training is even more impressive.
[ Parent | on HN ]