News for hackers. Stuff that matters.

thushanfernando

Re:Rust–: Rust without the borrow checker on Thursday January 01, 2026 @07:18AM (#46453519)
Bust?

Show HN: Olla – Lightweight LLM Proxy for Homelab and OnPrem AI Inference

We’ve been running distributed LLM infrastructure at work for a while and over time we’ve built a few tools to make it easier to manage them. Olla is the latest iteration - smaller, faster and we think better at handling multiple inference endpoints without the headaches.The problems we kept hitting without these tools:* One endpoint dies > workflows stall* No model unification so routing...

( Read More... | no comments | on HN )
It's not down - just following an eventual consistency model :D refresh again in a few hours. Otherwise he's probs doing a mass refactor... into 37 independently failing microservices.
The Blog Post from Google for context and background: https://blog.google/outreach-initiatives/entrepreneurs/ai-na...

Older »