News for hackers. Stuff that matters.
Re:GPUs for Deep Learning in 2023 – An In-depth Analysis by ipsum2 on Wednesday January 18, 2023 @08:07PM (#34435442)
It's pretty clear from the article that 8-bit inference is referring to FP8, as the author claims int8 is similar to fp16 inference performance.

> Ada/Hopper also have FP8 support, which makes in particular 8-bit training much more effective. I did not model numbers for 8-bit training because to model that I need to know the latency of L1 and L2 caches on Hopper/Ada GPUs, and they are unknown and I do not have access to such GPUs. On Hopper/Ada, 8-bit training performance can well be 3-4x of 16-bit training performance if the caches are as fast as rumored. For old GPUs, Int8 inference performance for old GPUs is close to the 16-bit inference performance.

There are no comments on this comment yet.