Llama 3.1 Official Launch

Re:Llama 3.1 Official Launch by AaronFriel on Tuesday July 23, 2024 @10:59AM (#41046669)

If previous quantization results hold up, fp8 will have nearly identical performance while using 405GiB for weights, but the KV cache size will still be significant.

Too bad, too, I don't think my PC will fit 20 4090s (480GiB).

[ Parent | on HN ]

Re:Llama 3.1 Official Launch by knicholes on Tuesday July 23, 2024 @11:21AM (#41046925)

I've got a motherboard that will support 8!

[ Parent | on HN ]

Re:Llama 3.1 Official Launch by Zambyte on Tuesday July 23, 2024 @11:47AM (#41047292)

40,320 4090s?? What witchcraft is this?! :D

[ Parent | on HN ]

Re:Llama 3.1 Official Launch by sebastiennight on Tuesday July 23, 2024 @07:05PM (#41051952)

All the more impressive when you realize that Groq's infrastructure (based on LPUs) was built using only 6!

[ Parent | on HN ]