Re:Run 70B LLM Inference on a Single 4GB GPU with This New T...
by alchemist1e9 on Sunday December 03, 2023 @05:35PM
(#38511453)
Does any one know the amount of data that is passed between layers? specifically the data size as this information determines the viability of distributed inference.
[ Parent
| on HN ]
There are no comments on this comment yet.