Ask HN: Pros and cons of switching to self-hosted inference?
Management is pushing us toward running open-weight models in-house after some compliance conversations around data privacy. Before we commit, we'd love to hear from people who've made this transition.Specifically curious about:Did it actually end up cheaper than paying for API access at your request volume? Were there any issues related to managing performance, more specifically latency,...