Re:Cost Per 1M tokens Of Self Hosting Llama-3
by speakspokespok on Friday June 14, 2024 @03:34PM
(#40684200)
Why did this only occur to me recently? You can selfhost a k8s cluster and expose the services using a $5 digital ocean droplet. The droplet and k8s services are point-to-point connected using tailscale. Performance is perfectly fine, keeps your skillset sharp, and you’re self-hosting!
[ Parent
| on HN ]
Re:Cost Per 1M tokens Of Self Hosting Llama-3
by Helithumper on Friday June 14, 2024 @04:56PM
(#40685035)
You can also just directly connect to containers using Tailscale if it's just for internal use. That is, having an internally addressable `https://container_name` on your tailnet per-container if you want. This way I can setup Immich for example and it's just on my tailnet at `https://immich` without the need for a reverse proxy, etc...
Re:Cost Per 1M tokens Of Self Hosting Llama-3
by SparkyMcUnicorn on Friday June 14, 2024 @06:05PM
(#40685576)
And you can use Tailscale Funnel to serve it publicly. No need to pay for a cloud instance.
Re:Cost Per 1M tokens Of Self Hosting Llama-3
by LorenzoGood on Saturday June 15, 2024 @10:50PM
(#40694327)
I essentially do this with my homelab.