On AMD, if I'm not not mistaken, FP16 is twice of FP32, as it should be
"For their consumer cards, NVIDIA has severely limited FP16 CUDA performance. GTX 1080’s FP16 instruction rate is 1/128th its FP32 instruction rate, or after you factor in vec2 packing, the resulting theoretical performance (in FLOPs) is 1/64th the FP32 rate, or about 138 GFLOPs."
https://www.anandtech.com/show/10325/the-nvidia-geforce-gtx-...
"FP16 performance is 1/64th and FP64 is 1/32th of FP32 performance."
https://blog.inten.to/hardware-for-deep-learning-part-3-gpu-...
I don't understand this meme. The consumer cards are different chips with slow fp64 hardware. In what sense is that "limiting" performance relative to the enterprise cards?