News for hackers. Stuff that matters.

A comment on Tokens are a big reason today's generative AI falls short. Parent comment

Re:Tokens are a big reason today's generative AI falls short by bubblyworld on Saturday July 06, 2024 @05:30PM (#40893466)
Why would you think that's a walk in the park? Have you actually tried formalising stuff in Lean/Coq? I have, and even with a postgraduate maths degree behind me it's hard as hell!

The fact that Kevin and his team are formalising FLT is incredible, but they all have decades of experience with this stuff (!!).

Transformers can do arithmetic (and many other things) just fine, do a bit of searching on arxiv and you'll find papers from 2023 showing that nano-scale transformer models suffice. It really is a data problem, not a fundamental limitation with the technology.

[ Parent | on HN ]
Re:Tokens are a big reason today's generative AI falls short by soloist11 on Saturday July 06, 2024 @05:34PM (#40893492)
What is your degree in?
[ Parent | on HN ]
Re:Tokens are a big reason today's generative AI falls short by bubblyworld on Saturday July 06, 2024 @06:01PM (#40893672)
Master's studying representation theorems in nonmonotonic logic, left academia for industry during my PhD. Fun spaced out maths problems. I tried to formalize my thesis in Lean but it is nowhere near as simple as you make it out to be.
[ Parent | on HN ]
Re:Tokens are a big reason today's generative AI falls short by soloist11 on Saturday July 06, 2024 @11:09PM (#40895002)
Maybe that's because you're not generally intelligent.
[ Parent | on HN ]