Training on FPGAs starts to make a lot of sense when you consider low bit precision computation (e.g. DoReFa-Net).