Re:Large language models are having their Stable Diffusion m...
by dheera on Monday March 13, 2023 @05:28PM
(#35143034)
Conceptually that's already the case i.e. L1 cache -> L2 cache -> RAM -> swapfile
If the definition of RAM is DIMMs then it's really just a matter of merging the CPU and GPU into one unit with a massive L2 cache (or maybe we have an L3 cache) of e.g. 32GB that's able to hold a massive model without cache misses.
If the definition of RAM is DIMMs then it's really just a matter of merging the CPU and GPU into one unit with a massive L2 cache (or maybe we have an L3 cache) of e.g. 32GB that's able to hold a massive model without cache misses.