OpenAI cofounder Ilya Sutskever says the way AI is built is about to change

Re:OpenAI cofounder Ilya Sutskever says the way AI is built ... by theptip on Saturday December 14, 2024 @11:44AM (#42418086)

For a “legal LLM” you need three things: general IQ / common sense at a substantially higher level than current, understanding of the specific rules, and hallucination-free recall of the relevant legal facts/cases.

I think it’s reasonable to assume you can get 2/3 with a small corpus IF you have an IQ 150 AGI. Empirically the current known method for increasing IQ is to make the model bigger.

Part of what you’re getting at is possible though, once you have the big model you can distill it down to a smaller number of parameters without losing much capability in your chosen narrow domain. So you forget physics and sports but remain good at law. That doesn’t help you with improving the capability frontier though.

[ Parent | on HN ]

Re:OpenAI cofounder Ilya Sutskever says the way AI is built ... by pas on Saturday December 14, 2024 @05:28PM (#42419923)

And then your Juris R. Genius gets a new case about two Red Socks fans getting into a fight and without missing a beat starts blabbering about how overdosing on too much red pigments from the undergarments caused their rage!

[ Parent | on HN ]

Re:OpenAI cofounder Ilya Sutskever says the way AI is built ... by theptip on Saturday December 14, 2024 @07:36PM (#42420550)

Yeah, I think for the highest-value activities (eg legal advice) you expect to run the full frontier model.

But maybe you want to run a smaller one locally on your iPhone for privacy and accept the capability loss.

[ Parent | on HN ]