News for hackers. Stuff that matters.

A comment on Run 70B LLM Inference on a Single 4GB GPU with This New Technique. Parent comment

Re:Run 70B LLM Inference on a Single 4GB GPU with This New T... by onlywicked on Sunday December 03, 2023 @02:48PM (#38510093)
There is already an implementation along the same line using the torrent architecture.

https://petals.dev/

[ Parent | on HN ]

There are no comments on this comment yet.