News for hackers. Stuff that matters.
Re:Show HN: Arch-Router – 1.5B model for LLM routing by pref... by adilhafeez on Tuesday July 01, 2025 @08:25PM (#44439208)
Short answer is latency impact is very minimal.

We use envoy as request handler which forwards request to local service written in rust. Envoy is proven to be high performance, low latency and highly efficient on request handling. If I have to put a number it would be in single digit ms per request. I will have more detailed benchmark in the coming days.

There are no comments on this comment yet.