Re:Nvidia Was in Trouble Before the Jensen Huang Mess
by justinc-md on Monday November 16, 2026 @02:14PM
(#50521667)
What's your opinion of high bandwidth flash? It seems ideal for storing and retrieving model weights. Do you see that dramatically changing inference chip design over the next couple of years?
Re:Nvidia Was in Trouble Before the Jensen Huang Mess
by jpgvm on Monday November 16, 2026 @09:28PM
(#50526649)
I think anything that doesn't compete with HBM for wafers will get it's day in the sun somewhere.
I don't know if HBF in particular is going to be huge but it's just so hard to predict the future in this space.
My personal opinion is we should be paying attention to SerDes and base dies to keep compute flexible and able to adapt to whatever memory and storage solutions are not supply constrained.
Re:Nvidia Was in Trouble Before the Jensen Huang Mess
by buildbot on Monday November 16, 2026 @03:26PM
(#50522718)
In my personal opinion as well "specialized enough" chips are exactly the right way to go - though disclaimer, also At Nvidia, working on spec dec/external drafts :)
With things like https://developer.nvidia.com/blog/co-designing-ai-models-usi... you can specialize one chip for decode/drafting, and the other for prefill and draft verification.
Re:Nvidia Was in Trouble Before the Jensen Huang Mess
by Melatonic on Monday November 16, 2026 @03:36PM
(#50522881)
Is LPX only going for rack size integration or is the plan to offer an LPX companion chip to something like a 6000 Blackwell ?
Could see a board with something like two SXM slots (maybe or similar) and one being for LPX optionally be cool
Specifically: https://www.nvidia.com/en-us/data-center/lpx/
We ran a pure-LPU demo for HotChips that AA benchmarked: https://developer.nvidia.com/blog/how-nvidia-groq-3-lpx-unlo...
TLDR: LPUs are very efficient, quite a lot more efficient and faster tok/s/u than anything else at this years Hot Chips.
It's my personal opinion that with the way models change you don't want silicon that is too specialised but it needs to be specialised enough. LPUs are designed to strike this balance such that they remain flexible if model architectures change drastically.