News for hackers. Stuff that matters.
Re:OpenAI unveils its first custom chip, built by Broadcom by nine_k on Wednesday June 24, 2026 @04:50PM (#48665456)
In very crude terms, AFAICT, if you have a bunch of matrix multiplications, but one of matrices (the one with model weights) doesn't change, you can seriously speed up the computation. One thing is that you don't need to re-fetch the elements of the constant matrix, you can keep it near the ALUs. Then you maybe can detect and ignore sparse / empty blocks by marking them once.

IDK how the custom hardware exploits this; would love to hear any ideas!

Re:OpenAI unveils its first custom chip, built by Broadcom by guyomes on Wednesday June 24, 2026 @05:30PM (#48665856)
> IDK how the custom hardware exploits this; would love to hear any ideas!

You might like this article [1], titled "FPGA-based CNN Acceleration using Pattern-Aware Pruning". More context and details can be found in the PhD thesis of Léo Pradels [2].

[1]: https://inria.hal.science/hal-04689673/document

[2]: https://theses.hal.science/tel-05021575v1/file/PRADELS_Leo.p...

Re:OpenAI unveils its first custom chip, built by Broadcom by cm2187 on Wednesday June 24, 2026 @07:51PM (#48666957)
Random thought. Once models stabilise, could you possibly hardcode the model in gates? Or are they too large for a single chip?
Re:OpenAI unveils its first custom chip, built by Broadcom by 8note on Wednesday June 24, 2026 @08:04PM (#48667089)
Re:OpenAI unveils its first custom chip, built by Broadcom by lsaferite on Thursday June 25, 2026 @12:14AM (#48668799)
Re:OpenAI unveils its first custom chip, built by Broadcom by jwHollister on Thursday June 25, 2026 @01:18AM (#48669252)
wow if they can get something like this working, what happens to all this infrastructure? Hyperscalers have to be assuming the lifespan of that stuff wrong considering the next gen will be 1000x more efficient.
Re:OpenAI unveils its first custom chip, built by Broadcom by otterley on Thursday June 25, 2026 @02:01AM (#48669548)
The question isn’t whether it works (it does); the question is whether there are buyers for hardware that is obsolete the day it ships. Models evolve much more quickly than hardware can keep up.
Re:OpenAI unveils its first custom chip, built by Broadcom by simondotau on Thursday June 25, 2026 @02:16AM (#48669658)
Presumably at some point the rapid progress of models will plateau, at least insofar as a model could be frozen in time and remain economically useful for the expected life of hardware. Especially if it comes with compelling benefits e.g. dramatically lower latency and/or dramatically higher performance per watt.

If you can build chips that could run one specific LLM 100x faster than anything else, it would have a use case that nothing else could match.

Re:OpenAI unveils its first custom chip, built by Broadcom by fragmede on Thursday June 25, 2026 @03:38AM (#48670195)
https://www.cerebras.ai/ is exactly that! Holy shit it's fast.
Re:OpenAI unveils its first custom chip, built by Broadcom by otterley on Thursday June 25, 2026 @08:15AM (#48672253)
Cerebras is not that. Cerebras isn’t tied to a particular model like Taalas is. The latter is even faster than Cerebras.
Re:OpenAI unveils its first custom chip, built by Broadcom by jwHollister on Thursday June 25, 2026 @08:22PM (#48680935)
Yeah at a certain point if inference is fast and cheap, do I care if it's the smartest? I can iterate in a tighter loop.
Re:OpenAI unveils its first custom chip, built by Broadcom by lsaferite on Thursday June 25, 2026 @09:43AM (#48673280)
Those taalus chips apparently run at 1/10 the power as the current SOTA GPU setups. If they can execute even partially on their plan, it'll be a literal game changer.
Re:OpenAI unveils its first custom chip, built by Broadcom by indigo945 on Thursday June 25, 2026 @04:21AM (#48670528)
One obvious use case is edge computing, such as in industrial applications that cannot tolerate the risk of a network link or cloud service going down. Even embedded use cases are possible, such as an image classifier model in a security camera.
Re:OpenAI unveils its first custom chip, built by Broadcom by cm2187 on Thursday June 25, 2026 @05:33AM (#48671096)
In fact any application where the task is stable and the model good enough to address that task. As you suggest, industrial applications where a robot must deal with variants of the same repetitive task. Or a military drone which needs to be jamming proof.
Re:OpenAI unveils its first custom chip, built by Broadcom by Someone on Thursday June 25, 2026 @06:52AM (#48671643)
> Or a military drone which needs to be jamming proof.

That, if used in war, I would think, would need the ability to be updated frequently. For example, your enemy might find out (say by running tests on hardware they captured from you) that painting some red paint in a particular shape (a smiley might even work) on their hardware prevented your drones from attacking them because it confuses that pattern with the Red Cross logo.

Re:OpenAI unveils its first custom chip, built by Broadcom by TeMPOraL on Thursday June 25, 2026 @10:05AM (#48673624)
You keep the "reasoning core" burned and play the cat-and-mouse game at the I/O edge. Enemy invents a smiley shield, your R&D figures out some filtering step that defeats this effect without compromising general image recognition. Then the enemy figures out a new trick, your R&D invents a countermeasure, and so on - point is, this can happen for a long time in layers on top of the core model. If the enemy invents some robust way to attack the core that cannot be filtered out, it's game over for that hardware, but that is a much more difficult task and might take longer than expected service time of a given batch of drones.
Re:OpenAI unveils its first custom chip, built by Broadcom by SoftTalker on Thursday June 25, 2026 @12:54PM (#48676176)
Sort of mirrors how biological organisms work. E.g. in a bird, the core functionality of knowing how to fly is burned in. Hunting food is probably a combination of experiential learning on top of instinctive behavior, and is somewhat adaptable to local conditions.
Re:OpenAI unveils its first custom chip, built by Broadcom by 542354234235 on Thursday June 25, 2026 @12:56PM (#48676203)
Those are really two different things. One is the computer vision that could be “hard coded” and the other is the image library, that would be updated regularly. Look at facial recognition. You can download and run a facial recognition LLM on your GPU that looks at a library of your personal photos. The LLM doesn’t change when it scans your photos for faces, it just writes the data associated with a “face” to whatever library. When you add a new picture, it adds that face data and compares it to the library for a match. The actual LLM never needs to change. It is the same as the one I downloaded and ran on my GPU for my photos. If it was written on chips we both bought and installed, it would work the same way.[1]

[1] Yes, this is a massive simplification

Re:OpenAI unveils its first custom chip, built by Broadcom by SwellJoe on Thursday June 25, 2026 @05:18AM (#48670998)
The models have to run on something or they're useless. They can't run on future hardware today, and people want to use models today. So, if hardware is obsolete the day it ships, we're all using obsolete hardware, and there's no alternative to that.
Re:OpenAI unveils its first custom chip, built by Broadcom by otterley on Thursday June 25, 2026 @08:20AM (#48672300)
Taalas encodes the model into the hardware itself. The two are inextricably coupled. It’s like buying a CNC router that can’t be reprogrammed to build anything other than a specific predetermined kitchen cabinet. And the model used inside is frozen many months before the hardware ships, since the process from tapeout to production takes that long.

In contrast, tomorrow’s models will typically run, although perhaps more slowly, on general-purpose inference hardware that was released today or even years ago.

Re:OpenAI unveils its first custom chip, built by Broadcom by lsaferite on Thursday June 25, 2026 @08:41AM (#48672492)
They are betting on fast release cycles coupled with much lower costs (purchase and operations) mixed with the ability to have dynamic fine tunes on top of the static model.
Re:OpenAI unveils its first custom chip, built by Broadcom by wrsh07 on Thursday June 25, 2026 @08:53AM (#48672624)
Right, but there exist problems that need to be routinely solved and can be solved on glm 5.2. is the model state of the art when it is published? No. But when it comes out you could optimize it and let your solver run forever for quite cheap, and that could be useful if the only problems you want it to solve (for cheap) are solvable by that model.

And the high water mark of what can be solved by open models will keep going up.

Re:OpenAI unveils its first custom chip, built by Broadcom by largbae on Thursday June 25, 2026 @09:33AM (#48673141)
There may be all sorts of stable use case models that this could be interesting for. Imagine permanent voice translation circuits at a tiny fraction of the current price, glasses that subtitle the world with long battery life.
Re:OpenAI unveils its first custom chip, built by Broadcom by fulafel on Thursday June 25, 2026 @11:49AM (#48675196)
Current accelerators (TPUs, various onchip NPUs) are something close to this. Systolic array is the estabilished computer architecture term for flowing data from computation to computation without the overhead of a register file or von Neumann bottleneck.