News for hackers. Stuff that matters.

dannyboland

Show HN: Sturnus, a latency-based LLM load balancer

https://github.com/sturnus-dev/sturnus

I built sturnus because I wanted the latency-based load balancing of an AI gateway but without a big framework selling an enterprise plan.It’s a single static binary that does one thing, aiming to avoid the large surface area and security issues that have plagued some LLM gateways.Traffic is distributed by LLM provider latency and availability. The share going to other providers acts as a...

( Read More... | no comments | on HN )
After the last LiteLLM compromise, I took my requirement (an OpenAI compatible endpoint routing across multiple providers based on latency) and built that as a small static Rust binary - https://github.com/sturnus-dev/sturnus

Now that AI makes it cheaper to build such tools in-house, the case for adopting a maximalist framework and trusting that entire supply chain is a lot weaker.

Re:Resources to Get Good at Soldering? on Saturday September 05, 2026 @08:55AM (#49576069)
The principle behind a lot of good advice here is “creating the conditions for solder to naturally go where you want it to”. Cleaning, tinning, preheating are all part of creating those conditions.

So the moment you find yourself fighting or forcing the solder, it’s better to step back and create the right conditions.

Bleak. This flat and fast structure assumes that ICs are all senior devs, able to be largely autonomous. Who supports the junior engineer who is struggling?

There’s a lot of talk about how AI shifted the job market away from junior roles, but companies had already removed the management capacity for them.

“Drop” is a loaded verb in a SQL context. “Add” or “paste” might avoid the raised eyebrow.
Re:Ask HN: What do you use to host side projects? on Sunday August 30, 2026 @04:36PM (#49502504)
I use Hetzner - reasonable prices, EU based and one of the best benefits of AI is that running a pet server is no longer a real burden.
Re:Claude, Teach Me Something on Saturday June 06, 2026 @03:25PM (#48428101)
I've been using this loop for a while now and I'm impressed at the variety of what I've learned (from cooking to control loops).

I'd highly recommend it - the Socratic approach of putting me on the spot to guess is a refreshing break from the habit of just looking things up.

There looks to be some errors in the conversion from PDF -> web in this report. For example, the web version of Figure 7 has the legend colours reversed.
Would publishing code under a limited company be a viable approach to avoid liabilities from OSS?

Of course it shouldn’t be necessary, I’m just trying to think of a practical way for an individual to avoid this scenario.

Re:Building a Virtual Machine Inside ChatGPT on Saturday December 03, 2022 @06:47PM (#33848792)
This is a lot of fun!

I took a stab at doing something similar with virtualising an ML model for an arbitrary text classification task: https://mastodon.scot/@danny/109452332918910028

> I want you to act as a machine learning classifier. The training set consists of news topics and the label categories are “politics”, “celebrity” and “finance”. I will type text input and you will reply with what the classifier output would be. I want you to only reply with the output and nothing else. Do no write explanations.

Example training data:

Topic: Barack Obama

Label: politics

Topic: Kim Kardashian

Label: celebrity

Topic: Dow Jones

Label: finance

Topic: Tony Blair

Label:

Re:How to create a Python package in 2022 on Thursday July 28, 2022 @03:43AM (#32261504)
Yeah, I adopted the “src” dir layout after reading this post some time back:

https://blog.ionelmc.ro/2014/05/25/python-packaging/#the-str...

It describes some of the outdated motivations for the other layouts commonly seen with python, as well as the many benefits of the “src” layout.

Re:How to create a Python package in 2022 on Thursday July 28, 2022 @03:12AM (#32261337)
This is almost exactly how I set up python projects; it’s reassuring to see it set out in one place.

I started using tox-poetry-installer[1] to make tox pick up pinned versions from the lock file and reuse the private package index credentials from poetry.

[1] https://github.com/enpaul/tox-poetry-installer

I recommend reading the Tao Te Ching for some ancient advice on getting out of the rat race mindset:

https://github.com/lovingawareness/tao-te-ching/blob/master/...

Excess wealth just means more things to lose.