No. The hard part is how to deploy without cloud and without GPU.

Many papers show how to quantize weights from float to binary for super efficient and faster inference, but nobody did open source their deployment ready ARM CPU kernels and FPGA HDL code on how to do so.

What? Surely deployment is by far the easiest part of machine learning? The learning is the hard part. If you are resource constrained, then learning a small/efficient model is even harder, obviously. As for binary math, usually all you need is the standard bitwise operators and popcount.