Specifically: https://www.nvidia.com/en-us/data-center/lpx/
We ran a pure-LPU demo for HotChips that AA benchmarked: https://developer.nvidia.com/blog/how-nvidia-groq-3-lpx-unlo...
TLDR: LPUs are very efficient, quite a lot more efficient and faster tok/s/u than anything else at this years Hot Chips.
It's my personal opinion that with the way models change you don't want silicon that is too specialised but it needs to be specialised enough. LPUs are designed to strike this balance such that they remain flexible if model architectures change drastically.
I don't know if HBF in particular is going to be huge but it's just so hard to predict the future in this space.
My personal opinion is we should be paying attention to SerDes and base dies to keep compute flexible and able to adapt to whatever memory and storage solutions are not supply constrained.
With things like https://developer.nvidia.com/blog/co-designing-ai-models-usi... you can specialize one chip for decode/drafting, and the other for prefill and draft verification.
Could see a board with something like two SXM slots (maybe or similar) and one being for LPX optionally be cool
I guess they mean Groq. Anyone know what nvidia did with groq btw?