AI is now capable of developing its own inference hardware

Re:AI is now capable of developing its own inference hardware by random__duck on Tuesday October 06, 2026 @02:10PM (#49982005)

Opened the RTL, looked at the floating point math, learned that apparently you don't need correct floating point operations for LLMs, closed the page.

[ Parent | on HN ]

Re:AI is now capable of developing its own inference hardware by fsbonetto on Tuesday October 06, 2026 @02:32PM (#49982261)

Indeed there are some bugs/"non standard behavior" regarding very small or very big floating point values. All of those where proven harmless for LLM inference. Thanks for pointing out the bug. If you caught something outside of that, please point that out so I can fix it ;)

[ Parent | on HN ]

Re:AI is now capable of developing its own inference hardware by pamcake on Tuesday October 06, 2026 @07:39PM (#49985770)

> All of those where proven harmless for LLM inference.

Could you share link(s) to those proof(s)?

[ Parent | on HN ]

Re:AI is now capable of developing its own inference hardware by fsbonetto on Tuesday October 06, 2026 @09:42PM (#49986894)

Yup, the tests already compare to a CPU hugginsface implementation, but I'm working on making running and verifying the results of those tests easier and more available. Also, working on fixing those bugs and make the code more reliable now that the underling hardware has been saturated (memory bound)

[ Parent | on HN ]

Re:AI is now capable of developing its own inference hardware by mitxela on Tuesday October 06, 2026 @02:35PM (#49982300)

Well, you don't. That's why 1.58-bit (ternary) quants are often used. But if that's what they're going for, no need to dress it up in a floating-point facade.

[ Parent | on HN ]