News for hackers. Stuff that matters.

A comment on Behind the curtain: what it feels like to work in AI right now. Parent comment

Re:Behind the curtain: what it feels like to work in AI righ... by staunton on Friday April 07, 2023 @07:50AM (#35480624)
Significantly higher latency than you have within a single datacenter. Think "my GPU working with your GPU".
[ Parent | on HN ]
Re:Behind the curtain: what it feels like to work in AI righ... by alexvoda on Friday April 07, 2023 @08:28AM (#35480893)
There are already LLMs hosted across the internet (Folding@Home style) instead of in a single data center.

Just because the swarm infrastructure hosting an LLM has higher latency across certain paths does not make it a swarm of LLMs.

[ Parent | on HN ]
Re:Behind the curtain: what it feels like to work in AI righ... by staunton on Friday April 07, 2023 @08:34AM (#35480940)
> There are already LLMs hosted across the internet (Folding@Home style)

Interesting, I haven't heard of that. Can you name examples?

[ Parent | on HN ]
Re:Behind the curtain: what it feels like to work in AI righ... by alexvoda on Friday April 07, 2023 @08:48AM (#35481046)
I read about Petals (1) some time ago here on HN. There are surely others too, but I don't remember the names.

1. https://github.com/bigscience-workshop/petals

[ Parent | on HN ]