Not true, pods can be gracefully shutdown, see https://cloudblog.withgoogle.com/products/gcp/kubernetes-bes...
> The clustering protocol also seems to assume that all members have stable identities and IP addresses, but that's obviously not the case in Kubernetes, where each new pod has a new identity and new IP address.
Both not true: Orleans cluster managed by a dynamic membership table which could be implemented by various tools(azure table, zookeeper, Kubernetes crds(etcd internally), etc), so nodes don't need static ip addresses also, in Kubernetes, you can deploy application as statefulset, by this way, pods will have stable identities
> Orleans was introduced by one person who has a background in distributed computing. Unfortunately, he's the only one in the organization with such a background. After he left, nobody could debug clustering problems.
Need source and proof for this one
Please don't post ridiculous opinions about things you are not familiar with, both Kubernetes and Orleans.
I have built successful products by Orleans & Kubernetes since two years ago, and now Orleans get better integration for Kubernetes than that time
edit: formatting & typo
> Early in the start-up process and at runtime, the silo will probe Kubernetes to find which silos do not have corresponding pods and mark those silos as dead [1]
This is an experimental fix in a MR [1] that comes 6 years after Microsoft announced support for Kubernetes [2].
> Orleans was created at Microsoft Research and designed for use in the cloud. [0]
I read: MS research developed the concepts of .NET Orleans 10 years ago without having Orchestrators like Kubernetes in mind. Now as an afterthought they have to come up with some patches for it not to cripple on short-lived Kubernetes nodes.
A proper Operator & Helm chart was requested in nov 2017 [3] and the issue still is open. I get the impression that it is either not possible to write a proper Kubernetes Operator without a major rewrite of .NET Orleans or Microsoft has a commercial interest for .NET Orleans not being a first class citizen on Kubernetes.
Whichever it is, I’m not going near a Kubernetes cluster hosting .NET Orleans.
[0] http://dotnet.github.io/orleans/Documentation/index.html
[1] https://github.com/dotnet/orleans/pull/6707
[2] https://azure.microsoft.com/en-us/blog/azure-collaboration-w...
If you want to discuss, feel free to message me on Twitter (https://twitter.com/reubenbond) or Gitter (https://gitter.im/dotnet/orleans).
- why it took up to 6 years for MS to release an experimental fix [1] to run on Kubernetes?
- where I can find the outstanding issue in the release notes that [1] tries to fix?
- why Microsoft doesn't write and release an Operator & Helm chart, instead asking the community?
- if .NET Orleans runs fine on Kubernetes, why is there a need for an experimental fix?
There is no experimental fix, just improvements. Things which can make life easier for developers running on Kubernetes by automating some things (setting addresses), and taking advantage of information that's available in a Kubernetes cluster (whether or not a pod has been deleted) and feeding that into the cluster membership system.
The reason the latter is useful is that it addresses something which can occur during initial dev/test, but which does not come up in production cases: when an entire cluster is deleted and redeployed with the same identity, the new instances try to contact defunct instances for a few minutes as a safety measure. The enhancement is to query Kubernetes to determine if it's worth trying to contact those nodes, or whether they're almost certainly dead.
- why Microsoft doesn't write and release an Operator & Helm chart, instead asking the community?
Helm charts aren't something we see requested often. Perhaps because Orleans is a framework which is embedded into the developer's application and not a service which gets deployed and stands alone (compared to, for example, a database). There are no separate Orleans pods, just the user's application pods. Internal users have been building applications on Kubernetes with their own Helm charts. Microsoft is not asking anybody to create those things, or anything, unless they want them for themselves.
However, Orleans doesn't seem to play very well with Kubernetes. Kubernetes pods can be uncleanly terminated at any time for any reason. Orleans really doesn't like unclean shutdowns: it will leave behind dead entries in the membership tables, which then cause problems in the future because new nodes would contact dead members and run into errors.
The clustering protocol also seems to assume that all members have stable identities and IP addresses, but that's obviously not the case in Kubernetes, where each new pod has a new identity and new IP address. This can cause members to fail uncleanly, which in turn pushes the clustering protocol into an infinite loop.
Orleans was introduced by one person who has a background in distributed computing. Unfortunately, he's the only one in the organization with such a background. After he left, nobody could debug clustering problems.