Re:Ask HN: How can ChatGPT serve 700M users when I can't run...
by nodja on Saturday August 09, 2025 @06:42AM
(#44845466)
Loading here refers to loading from VRAM to the GPUs core cache, loading from VRAM is extremely slow in terms of GPU time that GPU cores end up idle most of the time just waiting for more data to come in.
[ Parent
| on HN ]