News for hackers. Stuff that matters.

Shark1n4Suit

Show HN: Coelanox – auditable inference runtime in Rust (BERT runs today)

https://www.coelanox.com/

PyTorch and ONNX Runtime tell you what came out. They can't tell you what actually ran to get there — which ops executed, in what order, on what inputs.A model gets packaged into a sealed .cnox container. SHA-256 is verified before a single op executes. Inference walks a fixed plan over a minimal opset. Every run can emit a per-op audit log: op type, output tensor hash, output sample —...

( Read More... | no comments | on HN )
That's the key question. It feels like many of these tools are trying to solve a systems-level problem (cross-language model execution) at the application layer (with a Python library).

A truly universal solution would likely need to exist at a lower level of abstraction, completely decoupling the application's language from the model's runtime. It's a much harder problem to solve there, but it would be a huge step forward.