Re:Bringing K/V context quantisation to Ollama
by huijzer on Thursday December 05, 2024 @12:56AM
(#42325371)
I have Open WebUI on a Hetzner instance connected to Deep Infra. Works on mobile by turning the web page into an app. I find the web framework that WebUI uses quite bloated/slow, but apart from that it does work reliably. Price at Deep Infra is typically about $0.04 per month even when actively asking lots of questions during programming.
[ Parent
| on HN ]
There are no comments on this comment yet.