> as far as I've heard from HFT people Java's relatively popular in HFT

Java's popularity depends how HF in HFT is and the threshold for pain developers are willing to go through to develop Java in a way to avoid GC and reduce memory use (and hence reduce cache misses)

Deploying to Linux boxen in colocation facilities is the norm and there is little/no culture of using dotNet on Linux. We all know it exists and it's open source now, but most Linux developers don't reach for dotNet.

>and there is little/no culture of using dotNet on Linux. We all know it exists and it's open source now, but most Linux developers don't reach for dotNet.

it's pretty weird sentence "most Linux developers dont use dotNet", but on the other hands dotNet (Core) developers definitely do use Linux - I have dotNet things on Linux on prod for 2 years atm

Just because some people do a thing doesn’t mean most people do that thing. Most Linux devs don’t use .NET. Maybe they have technical reasons or maybe they just don’t want to use a Microsoft product

most people don't do HFT programming either

Right, so wouldn't intersecting those groups be even harder?

This sort of sounds like 2 clashing sets of anecdotal data.

Deployed a small internal .NET Core app on Linux recently. So far it's worked great! I've done a few major version package updates as well without issue.

> but most Linux developers don't reach for dotNet

On that note, how bad is .NET experience on Linux vs. on Windows? Besides the IDE (MonoDevelop is no Visual Studio).

I was about to dig into this topic soon - I'm particularly interested in how useful PowerShell is on Linux these days. On Windows, I appreciate the deep .NET interop it offers; I think it's closest experience to Lisp Machines that you can find in mainstream computing.

EDIT: Big thanks to the commenters who mentioned Rider, I didn't realize JetBrains had an IDE for .NET!

That said, to clarify my comment above, I'm more interested in the experience with the platform itself - how does .NET Core work on Linux in terms of performance, fragility, access to first-party and third-party libraries? Can I expect non-UI .NET code to port well to Linux? And, in PowerShell, do I get to do things like Add-Type "<insert bunch of C# code here>", or $foo = [Some.dotNET.Type]::new()?

I've been full time .NET Core on Ubuntu since Jan 2020. With the occasional booting a windows VM for a legacy project.

JetBrains Rider is such a pleasure to work with.

>Can I expect non-UI .NET code to port well to Linux

If it wasn't written with tons of pinvoke to win32 or using Windows-only services (like WCF), then sure.

I've been building NET Core applications for Linux and exclusively on Linux for a few years now, mostly using Rider as the IDE. No complaints. I haven't seen any Windows-exclusive libraries worthy of any attention in all that time. Even libraries with tons of native code (like imagemagick) have Linux support.

The official MS tooling is behind what they offer on windows, but it's getting there. You can collect GC and memory dumps

https://docs.microsoft.com/en-us/dotnet/core/diagnostics/dot...

https://docs.microsoft.com/en-us/dotnet/core/diagnostics/dot...

traces:

https://docs.microsoft.com/en-us/dotnet/core/diagnostics/dot...

and here's the port of the veritable sos tool:

https://docs.microsoft.com/en-us/dotnet/core/diagnostics/dot...

I think people on Linux go for Rider/VS Code instead of MonoDevelop

It's been rock solid for us. We're not using PS on Linux, just running/building binaries.

Rider is great and surpasses Visual Studio in my opinion.

I use mainly MacOs with VS Code (it’s no match for the real VS) and deploy dotnet apps in Linux containers (alpine). It’s a small app which more than an hundred users and it’s run fine for a year. I never add issues with libraries only available for Windows.

Its first class these days. You have native linux powershell, sql server, .net 5 (which is 4th linux/core version), vscode/rider or use visual studio with windows subsystem for linux.

Java being big in HFT is an accident of history, not due to sound engineering choices.

- Multiple implementations from various vendors

- More support for UNIX and mainframe flavours

- Given the multiple implementations, there are plenty of GC, JIT and tuning options to chose from

- .NET Core is the Python 3 of .NET world, not everything from .NET Framework is available or portable to non-Windows platforms

- Several JIT optimizations quite common in JVM world are only now coming to CLR, like dynamic rewrite of native code

Just that in difference to Python, everyone wants but not everyone can migrate to .NET Core. Also the source code is compatible.

Also, .NET 5 is now here, and 6 is already in preview, which are much, much more backwards compatible than .NET Core 3. And unlike Python, we had .NET Standard to start migrating code between Framework and Core. A lot of extant code on NuGet has already migrated and there is really very little you can't do these days.

Unless one is using third party controls, CMS like SiteCore, SharePoint and Dynamics, writing custom .NET code to be called by SQL Server stored procedures, having a working WCF infrastructure, using RDMS with EF 6 that isn't SQL Server, in-house stuff that isn't fully covered by the Windows Compatibility Pack, and a couple of other little things.

Also asp.net webforms is not supported, webservice/soap are 'supported' but the tooling is broken and i don't think you could extend a system with it in modern VS. The winforms editor works but is buggier with each new version, and asp.net mvc projects are similar but changed, and I'm sure the list of issues continues.

Also from my experience coaching and consulting there's a significant learning curve for .NET framework developers to work in the new core/.NET 5 systems.

So keep using .NET Framework. v4.8 is only 3 years old and there is no end date set on the support yet. Considering the last version of the v3 era will be supported until 2029, I'm sure the last version of the v4 era will get quite a long period of support as well.

https://docs.microsoft.com/en-us/lifecycle/products/microsof...

I bit the bullet and transitioned from WebForms to ASP.NET Core. I think it's easier than WebForms ever was. You can replicate 90% of what WebForms did with Razor Pages, but you also get much easier support for building RESTful APIs. And with DI built-in, I don't have to worry about junior devs forgetting to close/dispose database connections ever again.

On my world those decisions come with project budget, if no department puts the money to hire the consultants that would do the job, there is no transition.

What do you want me to say? Because your organization is stuck in the past, that means .NET can't move forward?

If you've followed the .NET blogs at all, they have very good reasons they haven't prioritized support for these legacy systems. There isn't some cabal of platform designers in Redmond scheming on how to screw over developers at ossified orgs.

Not sure why I'm bothering. Everytime someone mentions .NET on here, someone comes out of the woodwork to complain about their pet features not being supported. Actually, come to think of it, it's often you in particular. We're going from .NET Framework being fundamentally tied to Windows to a new .NET that runs on everything. ASP.NET was built on HTTP.sys, a Windows driver for a web server. I don't know, is it terribly surprising that trying to migrate .NET off of Windows-only dependencies is going to break things downstream?

But if you still need this stuff that only runs on Windows, well, you got at least decade of support to look forward to.

I literally think you're asking for too much.

It is not me that is asking, rather the Fortune 500 companies where software development is a cost center, that hire us to keep their production systems and factory lines rolling.

On my pet projects I use whatever I feel like.

Not even Microsoft teams themselves have done it, plenty of MS stuff like SharePoint, Dynamics and SQL Server rely on .NET Framework.

Source being compatible doesn't help when the libraries require code rewrites anyway.

SQL Server itself does not rely on .NET Framework. SQL CLR, an optional component does. We run SQL Server in Linux containers now.

MS SQL Server started out as a port (to OS/2!) of Sybase SQL Server, which ran on Xenix (among other unixes). History doesn't repeat, but it does rhyme.

Try to run SQL Server stored procedures that rely on SQL CLR.

Why? That's just asking for poor perf, even on Windows. I would never do that.

Leave CLR in applications, where it belongs.

Customers decide what gets done.

Actually its not that easy, depending which parts of framework you used.

It's certainly reasonable, but will take re-work for many applications beyond the project structure migration. I think in .NET 6/7/8 the .NET framework migration situation will improve to be mostly pure migration.

For example I migrated about 75% of a moderately large winforms system directly, but 25% needed rewrites due to missing core implementations for framework libraries. e.g. MS chart component not implemented in core yet, soap services changing to grpc, and various others. And this system was originally built to minimise third party dependencies and use the core framework or microsoft provided libraries wherever possible.

Python 3 broke the language, .NET Framework and Core differ on the libraries, that a very different thing.

When working code breaks in the new version it doesn't matter what broke, it still doesn't work.

Sure, but comparing to python is disingenuous. Python3 was a complete break from Python2. The first version of .net core (5ish? years ago) broke the standard library down to only libs that work cross platform.

Is there anything in .NET 5 you're missing from .net 4.x?

I already mentioned in a couple of replies, even Microsoft own products like SharePoint, Dynamics, SQL CLR aren't on Core.

And on the other hand, Bing is running on the first preview version of .NET 6.

MS is a big company, they aren't going to immediately migrate everything just like Google didn't migrate every single project to Go when they were heading it.

> what C# lacks here?

Options. You are talking about the very tiny subset of Java users who actively worry about the relative differences in performance tweakability of different JVM implementations. Chances are that they routinely run multiple VMs and VM configuration sets in parallel on real world inputs to find the fastest configuration.

"Look at this nice VM, trust us, it's very fast, rewrite all your code and see for yourselves!" just won't be very convincing to those. With that crowd, .NET isn't competing against the JVM, it's competing against against an entire market of competing JVMs.

Does anyone know of any HFTs using C#?

I don't know of any, but I have been building prototypes in .NET Core 3+ that give me confidence in using this platform as a basis for an ultra-low-latency business system.

The biggest issue would be avoiding heap alloc, but that is actually not too difficult if you manage your data well enough. Most transactions in this business can be defined in terms of structs that are <1kb in size.

One other approach I have started to look at is sacrificing an entire high-priority thread to handling well-batched transactions (or extremely important timers) so that I never have to yield to the OS. In HFT, you always get an entire physical server to yourself, so eating 1 out of 32 threads is not a huge deal. In testing of this idea, I have found that I am able to reliably execute timers with accuracy of around 1/10th of a microsecond.

Java probably has more optimizations around GC suppression, but I feel like avoiding allocation in the first place is the most important bit. I believe there are already some ways to trick .NET into not running GC.

Out of curiosity, what is the domain for the prototypes you're building?

For now, I have been using this to test a toy/custom 2d client-server framework. All client events are piped to the server and processed in 1 gigantic ring buffer. This allows for ridiculous amounts of throughput due to batching effect. I am also using recurring, high-precision timers scheduled per client for purposes of triggering redraws and other important events as appropriate. This allows for complete decoupling of the event handling and client rendering pipelines. The breakdown is something like:

1 thread for ultra-low-latency timer execution

1 thread for processing the actual client event ring buffer, producing a consistent snapshot after each microbatch execution.

14+ threads for servicing HTTP requests (i.e. enqueuing client events), timers and redrawing client views using the near-real-time snapshots of business state.

My thinking is that if I can build something in this domain that is satisfactory, I could consider bringing it to an HFT firm as well.

This[0] guy claims to work at a shop that uses C# for HFT. This[1] comment chain from a post about Java HFT has users claiming to use C# for HFT.

[0]: https://www.youtube.com/watch?v=ug_UC4lxMr8 [1]: https://news.ycombinator.com/item?id=24896616

Accenture used C# for a HFT shop but that shop went away from it to C++.

Java is not popular in HFT other than 2-3 shops using it.

C++ is best choice IMHO. It has become easy to use it and plus you can get compile time branch prediction with templates.

RAII also amazing. RAII looks like a GC but it's not but it's works like a GC ..

What do you mean by compile time branch predictions?

You pay cost for branch predictions at Runtime. You can avoid it at compile-time using Templates or even constexpr. .

I still don’t understand what you mean, please correct me where I’m wrong.

Branch prediction is a CPU level thing, and mis-predictions has a cost. You can do things like partitioning data beforehand so that a given if condition will take the same branch each time in the partition, but I don’t see how does it apply to templates at all.

You can’t avoid branches that depend on runtime infos, and those branches that depend on compile time known infos can be elided either automatically by the compiler if it can prove it is constant for example, or by things like constexpr, templates as you say. But it’s nothing too fancy, the dumb C-preprocessor macro can do similar things.

JIT-compiled languages can sometimes elide branches based on runtime data, so there is that.