Re:Petals: Run 100B+ language models at home bit-torrent style
by _joel on Monday January 02, 2023 @05:29AM
(#34216392)
If you read the article text and the response from the dev then yes, inference can happen at 1/s or if parallelised, more. I'm not sure what your parameters are for a realtime system. If you're talking about network reliability, that's a different issue. Yes it can infer quickly, can it do it reliably is another matter.
[ Parent
| on HN ]
There are no comments on this comment yet.