First-off, you mentioned micro-services. The mechanisms that support service and application versioning seem well-intentioned and well-designed. In many ways the mechanisms run counter to some of the key benefits of a micro-service architecture (especially surrounding decoupling development teams).
If your latency is high, deployment times out. We had to use a remote desktop session to a VM near the cluster to deploy manually (solved with CI/CD). This impedes teams who are trying it out, when they are remote.
It has a pretty hostile development experience. Before you can even open a project in Visual Studio, you need the entire SDK installed. "Which SF installer did you use?" was a frequent question in Slack, because there are a bunch of places where it resides.
Whenever you add a project to your solution, you have to ensure that it targets x64 (not AnyCpu). Strange compilation errors abound if you don't. We get it Microsoft: the runtime only supports x64 - you don't have to enforce it to this degree. AnyCpu runs just fine on x64. It sounds like nitpicking, but it gets horribly frustrating.
They do (or used to do) strange things their .targets files. It doesn't behave like the C# build system, with a very clear happy-path. I spent about a week working around $(MSBuildProjectDir) not working. I would fix this by making the entire thing nothing more than a Nuget package. Anyone can understand having to have an emulator installed to debug, but the project system is hostile.
It's a big pity because the tech is amazing. I've personally started championing Orleans+Docker as a more friendly alternative, but as I said: others on my team adore it.
Eventually ported to .net core and Orleans 2 on k8s and it's been a more pleasant experience.
Trying to get .net core app running on SF was extremely frustrating and confusing. K8s was a MUCH more natural experience.
I was really disappointed in the tooling. The sfproj files were this incomprehensible mess. If the ui didn't quite work you are hosed.
I would go on but I'm on mobile
It's a fair critique though. It's not obvious how you would build a proper SF app that takes advantage of the platform. Hopefully that will change as it is open sourced.
I built something on an early build of SF that was pretty simple and it was hard to get up to speed. It was more of a batch oriented thing that ran off of queue inputs, so I didn't have to deal with getting load balancing inbound traffic working or other stuff.
Looking at SF recently it looks like they have made a lot of interesting changes like DNS service resolution. Something like Azure AD managed service instances would make bootstrapping secret management a lot easier. I would really like to see the ability to write PowerShell that plugged into the SDK and could take advantage of everything.
The comparison to K8s is more like the "guest exe" project wrapper they had. If you aren't interested in the API's for plugging into their storage or other platform features you can just wrap an executable and specify some parameters about how to host it in configuration.
This is one case where the VS tooling didn't do SF any favors. People think it's the only way to use the thing.
But you could use `guest executable`, which has no dependency on their tooling. You can create exe with any language/stack.
But honestly, K8s has first class support from MS, Google, AWS. Has major buy-in from the community, good docs, plenty of recorded talks, presentations and war stories and a very active community.
SF has basically none of that. At the end of the day, SF isn't massively better than K8s for my use case and It would have to be to make up for all those disadvantages.
I think it's absolutely correct that the tooling for SF has been rough to use.
Especially at the beginning we were running into many issues related to the transitions between VS2015 and 2017, different project systems, changes in .NET/.NET Core, and our own lack of understanding about how the tools should work (I used to joke that I'm the PM that lives furthest from UI).
We had the luxury of being not-as-good at tools when we were just an internal thing at MS because other teams would pick up that slack. They were pretty forgiving, but that let us think that we had the development story figured out. We most definitely did not.
When the real VS org got involved before we announced the product publicly, it made a world of difference and I think some of us started to realize how much our previous experience sucked. It has taken us a while to learn how people actually want to develop on top of this thing in the real world and to climb out of that pit, building experience as we go.
In the last few months things have gotten better. There's still lumps (deploying many services to my dev box is still way slower than I'd like), but there's also been some improvements. We've made the transition to tools that are included in VS2017 by default (rather than having to download and maintain them separately), and made sure our APIs support .NETCore/Standard. Some of the MSBuild weirdness is still around (like whatever this is[1]) but overall the environment is much more stable and that's helping tremendously.
We've also made some SF specific changes around supporting what we call "Application Refresh Mode" and a single node local box experience (rather than making you run a whole 5 node cluster on your dev machine and doing full upgrades all the time, which can be more taxing). These things help speed up the hack-build-run dev cycle.
Most recently we created a whole new FabricClient[2] which is self contained and works over just http. This is a big change and fixes some of the things mentioned in this thread (like having to have the SDK somehow installed on your build machines just to deploy to a cluster).
Plenty of room left to improve in the tooling and improve the product overall. Thanks to folks for their comments - again this sort of feedback is really useful. I try to read the HN comments when we get our stuff posted here and pull out tidbits, but you can also just open up items in our GitHub issues list[3], which gets triaged fairly frequently.
[1]: https://github.com/Azure/service-fabric-issues/issues/1095
[2]: https://blogs.msdn.microsoft.com/azureservicefabric/2018/05/...
Of course, the stateful service feature is lost. But k8s does not have that feature either.
Till now, I see its `guest executable` solution a good balance.
Persistent Volume Claims go a long way toward the stateful service "feature".
To go another step further on stateful services, SF's stateful services are only supported in a couple languages. Where mounting a volume in K8s which will follow your container is pretty darn accessible to any language.
The biggest problem imo, is that so very few people are using it, at least who talk about it, that this might be the first mention I've seen outside of MSDN, and I started looking hard at it three years ago or more.
I'd start with Erlang. The track record of Erlang for those parameters is just about unbeatable. And then there is the very careful way in which they deal with backwards compatibility, which is a major asset for any long lived project. If your projects live (or are intended to live) for a decade or more such details really matter.
And, yes, I know... whatever feature I could name from Service Fabric has some sort of open-source thing for Kubernetes from someone that probably/almost/kind-of/sort-of/some-assembly-required does the same thing as Service Fabric. Or I could just use Service Fabric, and not play the Kubernetes equivalent of "what's in your node_modules directory?"
Microsoft has an existential-level bet on Service Fabric. It's 1) not going anywhere, 2) only getting better, 3) is already a better container orchestrator than Kubernetes, if that's all you want to use it for.