Please don't rely on my [inferior] explanation and search for yourself (start here for insance: https://www.google.com/search?q=what+is+compute+in+memory+an...). Asking about a new field is much better done with AI these days, because you can get it to explain just the interests and questions you personally have, rather than a generic and boring explanation.
In short, better than HBM is not having to move data in and out of memory at all or at least not as much. The compute part in inference is much simpler than the data hauling part.
Couple of examples: Check out Axelera's Europa board (upcoming) or their Metis board (existing). Or check out Mythic.ai which FINALLY managed to raise funds in Nov / Dec 2025. Ironically, her PhD thesis had been gathering dust before that.
I thought HBMs are technically the fastest possible thing for inference.