Yeah nowadays with post-training quantization techniques as well as things like squeezenet which is quantization aware training technique, models are becoming fairly small to be able to run on phone smoothly