sszuecs
I am one of the maintainers of https://opensource.zalando.com/skipper http proxy library, which can support similar cases. We use this at Zalando https://www.zalando.com/ in Kubernetes and allow developers to connect to different kind of data applications including chat based LLMs or notebooks. We have of course OTel/Opentracing support https://opensource.zalando.com/skipper/operation/operation/#....
Likely the comparison with lb algorithms round robin and least connections is not a fair choice. Better would be to compare with consistent hash, that naturally does stateful load balancing. In skipper you can tune the behavior by filters https://opensource.zalando.com/skipper/reference/filters/#co... and https://opensource.zalando.com/skipper/reference/filters/#co... per route.
You don't want auto scaling? You can also limit concurrent requests to a route with queue support and make sure backends are not overloaded using scheduler filters https://opensource.zalando.com/skipper/reference/filters/#sc....
If you need more you can also help yourself and use lua filters to influence these options https://opensource.zalando.com/skipper/reference/scripts/ .
We are happy to hear from you, Sandor
We add dns names for cluster internal communication through coredns templates to this api gateway. https://opensource.zalando.com/skipper/kubernetes/east-west-... You can adapt it step by step and stop without a bid all in approach as service mesh vendors promote.
If you have not enough you can get even more advanced and add hpa by requests per second.
We use all of these in >50 production clusters with regular shop and order traffic far beyond 10k requests a second. We are interested in features, but even more in stability. We are interested in the community and will fix most of the issues you will show us to make errors less likely happen for us, too. We don’t sell software nor support, that’s why we have less advertisements and don’t show up on every kubecon.
I hope to read you in our community channels: https://github.com/zalando/skipper/blob/master/readme.md#com...
From our benchmarks we can outperform nginx in the pure routing case, while nginx will outperform skipper in the pure sendfile case. HAproxy we did not tested, but I would bet it will be not different from the nginx case. IMO: in general use skipper as microservice router and nginx as streaming router that serves a lot of pictures or videos.
Feature wise we have a lot of things to add as HTTP router that you want to run in front of your microservices, please also see our docs: https://opensource.zalando.com/skipper/ Additionally we build a lot of Kubernetes integrations,for example https://github.com/zalando-incubator/stackset-controller which reduces boilerplate and enables you to do automated blue-green deployments and https://github.com/zalando-incubator/kube-metrics-adapter which enables you to do HPAs with custom metrics, for example hpa based on skipper route request per second.
Skipper serves well microservice workloads for Zalando and other companies, for example shopgun https://medium.com/@roffe/kubernetes-in-production-shopgun-2....