News for hackers. Stuff that matters.

A comment on Petals: Run 100B+ language models at home bit-torrent style. Parent comment

Re:Petals: Run 100B+ language models at home bit-torrent style by dpflan on Monday January 02, 2023 @08:37AM (#34217672)
I have not read the technical details, apologies for ignorance, but is there an opportunity for caching?
[ Parent | on HN ]
Re:Petals: Run 100B+ language models at home bit-torrent style by jerpint on Monday January 02, 2023 @12:25PM (#34220651)
Probably not, since you need to compute the activations of unknown inputs and there could be infinitely many variations of them
[ Parent | on HN ]