> Remark. It is important to acknowledge that previous research has highlighted the ability of neural networks to tolerate noise and even act as a regularization method. However, our proposed analysis differs in two ways. Firstly, it is the first comparison of its kind that focuses on pure 16-bit neural networks. Secondly, our analysis offers a unique quantification approach that distinguishes it from previous explanations related to regularization. Our approach examines floating-point errors and formalizes the concept of tolerance, allowing for its quantification as a measure of comparison with floating-point errors.

> Explanation of the Lemma.... in the worst case.... But as long as... the two classifiers M16 and M32 must have the same classification result on x.

So my rough understanding is that they're saying "Hey, it's not/not only because of regularization; it's because fp16 is really as good as fp32 for what we need it to do (but more efficient?"