ashishdhiman23

Nice results on the compression ratio. One thing worth measuring alongside file size is actual on-device inference behavior after quantization. We ran FP32 and INT8 variants of a similar-scale model on Snapdragon 8 Gen 3 through Qualcomm AI Hub. Inference barely changed (0.176ms vs 0.187ms) but peak memory actually went up slightly (121MB vs 124MB) which was counterintuitive. The file was 70% smaller but runtime characteristics didn't follow the same pattern. Would be interesting to see TFLite inference timings on actual mobile hardware alongside the size numbers. Compression without on-device profiling can be misleading.

The dynamic budget negotiation idea is interesting. One thing I've found working with on-device AI models is that the "budget" isn't just about what the device can handle in theory — it shifts meaningfully between runs on the same hardware. Ran MobileNetV2 100 times on a Snapdragon 8 Gen 3 and got 83% latency spread. Cold start was 2.689ms, steady state median was 0.369ms. The first couple of runs are basically a different device thermally. Curious how Composite Budget Contracts handle that kind of within-session variance. Do you re-negotiate after warmup or is the contract set once at initialization?