RAM now represents 35 percent of bill of materials for HP PCs

Re:RAM now represents 35 percent of bill of materials for HP... by seanmcdirmid on Thursday February 26, 2026 @01:47AM (#47162727)

LLMs need memory bandwidth to stream lots of data through quickly, not so much caching. Well, this is basically the same way that a GPU uses memory.

[ Parent | on HN ]

Re:RAM now represents 35 percent of bill of materials for HP... by zozbot234 on Thursday February 26, 2026 @06:21AM (#47164579)

OTOH, LLM inference tends to have very predictable memory access patterns. So well-placed prefetch instructions that can execute predictable memory fetches in parallel with expensive compute might help CPU performance quite a bit. I assume that this is done already as part of optimized numerical primitives such as GEMM, since that's where most of the gain would be.

[ Parent | on HN ]