With those latency requirements why did you use Java?

Plenty of people are writing low-latency trading applications in Java where the latency budget is under 100 microseconds. This has been the case for several years now.

It's by no means new or even especially difficult these days. The resulting code is far more maintainable and robust than C++ solutions.

I've had the opposite experience. For green-field applications, it's far easier to write the application in C++ than Java & meet the latency requirements. Java requires far too much tuning, where as C++, from the get go, you can generally just glance at the code and have a good idea of the latency.

We're currently fighting a Java app that in general has decent latency (10s of usecs), but has outliers of greater than a second when GC kicks in. We don't have that issue with the C++ components of our trading system.

It's always going to be subjective. If you have people experienced in gc tuning, it's pretty easy to weed those gc outliers out. I am currently doing that for one of the applications for the company I work for. For me, it would be far easier to write a low latency, high concurrency Java app than a c++ app. The last c++ app that I was asked to write works horribly and bleeds memory. But it was not an important app.

Oof. There's probably some compounding factors that make it more difficult, but usually getting GC pauses <500ms isn't that hard with current-gen collectors. Either being very careful with CMS, or just feeding G1 a lot of memory.

I’ve heard a pretty effective, lazy strategy for this scenario is to oversize the heap to avoid GCs through the trading day and to simply restart the application before the next day. Unsure if it’s viable for all trading platforms, but it makes sense for some I’m sure.

Depends on your order volume. Sure an oversized heap may work for low order volume, but if you're dealing with high volume, even hundreds of gigabytes of RAM isn't even for a trading day.

And yeah, we restart everyday (we only trade US equities, so plenty of downtime for a restart).

You would be surprised to know majority of trading companies/exchanges use Java for trading platform. So does majority of companies in finance sector. I have been working in this secor for over 10 years and have interviewed for many of them.

I would be surprised. I was under the impression that kdb+ holds the majority role in fintech.

If you used to tune JVM only, that might be interesting. Most often the Java code is what being tuned - to the point where there is no point to use Java - with (almost) no gc, off-the-heap memory allocations and all that jazz.

It's not either/or. Everything needs to be tuned to achieve the final latency requirements. This includes tuning java code to eliminate creating "garbage", immutable string objects mostly, unnecessary object creation, creating short lived objects compared to long lived etc. The other part right sizing jvm generational partitions, using right gc algo etc. Next part would be to find out the code path with lowest latency requirement and making it hot/jitting. Even next part would be to tune OS for things like thread pinning etc etc. You can go further and tune the network layer. Every little thing counts.

You can go even further and employ what's called mechanical sympathy (https://mechanical-sympathy.blogspot.com/) and write your code in a way that makes it easier for the processor to process faster.

I've never worked with application that use off-heap allocations or no-gc requirements.

This doesn't make much sense. Using object pools and and off-heap data structures doesn't warp the code that much, if at all. You still end up with pretty much all clean, simple Java code.

I've seen (10+ years old) low-latency Java apps where people avoid using any OO (ie everything is static methods)... and even then the code was still often easier to maintain and understand than equivalent C++ applications. Nobody does this any more anyways.