News for hackers. Stuff that matters.

A comment on Are OpenAI and Anthropic Losing Money on Inference?. Parent comment

Re:Are OpenAI and Anthropic Losing Money on Inference? by reliabilityguy on Thursday August 28, 2025 @01:40PM (#45054872)
It’s not that the fetching is the problem, but serving the data to many cores at the same time from a single source.
[ Parent | on HN ]
Re:Are OpenAI and Anthropic Losing Money on Inference? by supersour on Thursday August 28, 2025 @04:42PM (#45056813)
I'm not familiar with GPU architecture, is there not a shared L2/L3 data cache from which this data would be shared?
[ Parent | on HN ]
Re:Are OpenAI and Anthropic Losing Money on Inference? by reliabilityguy on Friday August 29, 2025 @01:28PM (#45066968)
MMU has a finite amount of ports that drive the data to the consumers. An extreme case: all 32 cores want the same piece of data at the same time.
[ Parent | on HN ]