News for hackers. Stuff that matters.
Re:Show HN: I've built a locally running Perplexity clone by openquery on Thursday April 04, 2024 @05:46AM (#39928327)
> I foresee that lots of inference would be happening on local machines with models being downloaded on demand

Why? It's much more efficient to have centralized special purpose hardware to run enormous models and then ship the comparatively small result over the internet.

By analogy, you don't have a search engine running on your phone right?

Re:Show HN: I've built a locally running Perplexity clone by Sammi on Thursday April 04, 2024 @05:59AM (#39928400)
You currently can't have a search engine running locally on your phone. Google search is possible the single largest c++ program every built. And nevermind the storage needs...

But in a few years we might be able to have LLMs running on our phones that work just as well if not better. Of couse as you mention the LLMs running on large servers might still be much more powerfull, but the local ones might be powerfull enough.

Re:Show HN: I've built a locally running Perplexity clone by vachina on Thursday April 04, 2024 @05:51AM (#39928358)
A more appropriate analogy would be driving your own car vs. taking the bus.
Re:Show HN: I've built a locally running Perplexity clone by bufferoverflow on Thursday April 04, 2024 @11:24PM (#39938401)
No, a more appropriate analogy would be driving your own billion-dollar super-yacht vs driving your own car.

Will not happen any time soon. Consumer hardware can't even run GPT-4 locally, and won't be able for a looong time. Each GPT-4 instance runs on 8 A100. The cost of such system is ~$81K. Not even in the ballpark of what most consumers can afford.

Re:Show HN: I've built a locally running Perplexity clone by dns_snek on Thursday April 04, 2024 @07:49AM (#39929134)
> Why?

Privacy, security, latency, offline availability, access to local data and services running on the device, just to name a few.

Re:Show HN: I've built a locally running Perplexity clone by ilc on Thursday April 04, 2024 @10:58AM (#39931268)
Big Tech + Countries: Those all sound like great reasons to centralize all access to AIs!