That defeats the whole purpose. You have no way of knowing that the version that the provider claims to be serving is the version that they are actually serving

You can always rent a box and serve an open-weight model to yourself remotely. That's probably the only economic solution that halfway makes sense, given hardware prices these days.

Anthropic released a paper on zero trust verification of inference running on the same model.

Everybody says they pin. Meanwhile, if they can swap in a lower quant and serve more throughput and make more pennies, they do. It's easy to say, "well, KLD is at 5% so come on that's pretty much the same". And then that's the new norm for a bit, and then comes more traffic and they say, well, shit, let's do that again, it's only another 5%...

You never know if youre really getting the offered quant. Only with local can you be sure.