News for hackers. Stuff that matters.
Re:Who uses Google TPUs for inference in production? by emu on Tuesday March 12, 2024 @07:38PM (#39686233)
> Such capability can’t even be contemplated on the TPU side because you basically get a black box.

I'll just leave this here: https://jax.readthedocs.io/en/latest/pallas/index.html

Re:Who uses Google TPUs for inference in production? by ein0p on Wednesday March 13, 2024 @01:45AM (#39688432)
Pallas is very new. Given how difficult these things are to debug and how half assed the XLA tooling generally is, I’d give it at least another year, if not two, before I touch it for anything prod related.