News for hackers. Stuff that matters.
Re:Locally hosted LLM use cases? by quickthrower2 on Tuesday June 27, 2023 @03:15AM (#36489827)
The advantages if local:

Privacy / Secrecy

Low Latency (as in network)

Low Latency (as in ttfb)

Can fine tune continuously and immediately.

No need for internet connection or just a “download only” type connection. Great for remote areas.

So where this might be useful?

A local NoT (network of things) that is privacy conscious.

An assistant for people who want to detox from being online.

A low latency code assistant for anyone who gets annoyed if they blink and nothing has happened yet.

A toy / pet type thing

LLM guidance for a submarine or space mission.

There are no comments on this comment yet.