News for hackers. Stuff that matters.
Re:Run 70B LLM Inference on a Single 4GB GPU with This New T... by alchemist1e9 on Sunday December 03, 2023 @05:35PM (#38511453)
Does any one know the amount of data that is passed between layers? specifically the data size as this information determines the viability of distributed inference.

There are no comments on this comment yet.