There has been a lot of research into using very low bit depth weights in neural nets, pruning, etc. I am pretty confident that this research, combined with purpose-designed silicon, will allow us to evaluate quite powerful neural net models on embedded systems.