News for hackers. Stuff that matters.
Re:Cost Per 1M tokens Of Self Hosting Llama-3 by Helithumper on Friday June 14, 2024 @04:56PM (#40685035)
You can also just directly connect to containers using Tailscale if it's just for internal use. That is, having an internally addressable `https://container_name` on your tailnet per-container if you want. This way I can setup Immich for example and it's just on my tailnet at `https://immich` without the need for a reverse proxy, etc...

https://tailscale.com/blog/docker-tailscale-guide

Re:Cost Per 1M tokens Of Self Hosting Llama-3 by SparkyMcUnicorn on Friday June 14, 2024 @06:05PM (#40685576)
And you can use Tailscale Funnel to serve it publicly. No need to pay for a cloud instance.

https://tailscale.com/kb/1223/funnel