News for hackers. Stuff that matters.
Re:OpenAI cofounder Ilya Sutskever says the way AI is built ... by losvedir on Saturday December 14, 2024 @09:19AM (#42417299)
That's kind of going in a different direction. The big picture is that LLMs have until this point gotten better and better from larger datasets alone. See "The Bitter Lesson". But now we're running out of datasets and so the only way we know of to improve models' reasoning abilities and everything, is coming to an end.

You're talking about fine tuning, which yes is a technique that's being used and explored in different domains, but my understanding is it's not a very good way for models to acquire knowledge. Instead larger context windows and RAG works better for something like case law. Fine tuning works for things like giving models a certain "voice" in how they produce text, and general alignment things.

At least that's my understanding as an interested but not totally involved follower of this stuff.

Re:OpenAI cofounder Ilya Sutskever says the way AI is built ... by kranke155 on Saturday December 14, 2024 @01:36PM (#42418650)
A human being doesn’t need to read the entire internet to pass the state bar.

Seems to me that need new ideas?

Re:OpenAI cofounder Ilya Sutskever says the way AI is built ... by lethibo on Thursday December 19, 2024 @04:15PM (#42465739)
Hi

Did you try eyelevel.ai It has made great strides in this Rag battle (ref the attached). Ready to discuss this further with you, and open an account on Ground x for you to test it: https://www.groundx.ai/

All the best Thibaut