Re:Cyphernetes: A Query Language for Kubernetes
by mltsd on Monday December 16, 2024 @04:04AM
(#42429270)
Pretty sure the apiserver just queries the etcd database (and maybe caches some things, not sure) but i guess it could be the apiserver itself that can't handle the data :P
Re:Cyphernetes: A Query Language for Kubernetes
by tasuki on Monday December 16, 2024 @05:25AM
(#42429737)
I no longer know anything about Kubernetes, but share your surprise! From first principles it seems the metadata should be small.
Re:Cyphernetes: A Query Language for Kubernetes
by nvarsj on Monday December 16, 2024 @07:23AM
(#42430387)
My knowledge is out of date now, but the main issues IMO are/were:
- No concept of apiserver rate limiting, by design. I see there is now an APF thingy, but still no basic API / edge rate limiting.
- etcd has bad scalability. It's a very basic, highly consistent kv store that has tiny limits (8GB limit in latest docs, with a default of 2GB). It had large performance issues throughout its life when I was using k8s, I still don't know if it's much better.
Re:Cyphernetes: A Query Language for Kubernetes
by crabbone on Monday December 16, 2024 @08:48AM
(#42430902)
Long ago I wanted to re-implement at least part of kubectl in Python. After all, Kubernetes has documented API... what I quickly discovered was that kubectl commands don't map to Kubernetes API. Almost at all. A lot of these commands will require multiple queries going back and forth to accomplish what the command does. I quickly abandoned the project... so, maybe I've overlooked something, but, again, my impression was that instead of having generic API with queries that can be executed server-side to retrieve necessary information, Kubernetes API server offers very specialized disjoint set of commands that can only retrieve one small piece of interesting info at a time.
This, obviously, isn't a scalable approach, but there's no "wrapper" you could write in order to mitigate the problem. The API itself is the problem.
Re:Cyphernetes: A Query Language for Kubernetes
by remram on Monday December 16, 2024 @10:16AM
(#42431788)
In my experience, they don't, you can just run more of them and you can stick them behind a load-balancer (regular HTTP reverse proxy). You can scale both etcd and apiserver pretty easily. Of course you have less control in cloud environments, I have less experience with that.
Re:Cyphernetes: A Query Language for Kubernetes
by alpb on Monday December 16, 2024 @11:29AM
(#42432539)
Kubernetes only lets you query resources by object type and that's only a prefix range scan on etcd database. There are no indexes whatsoever in the exhaustive LIST queries, and kube-apiserver handles serialization of the objects back and forth between multiple wire types. Over the years there has been a lot of optimizations, but you don't wanna list all pods in a 5000 node high density cluster every time you spin up client-side tools like this.
Is this a question of Kubernetes just sticking everything into "standard" datastructures instead of using a database?