News for hackers. Stuff that matters.
Re:Java is better than C++ for high speed trading systems by lmilcin on Thursday November 26, 2020 @01:15PM (#25222297)
Let me put my perspective on this.

As it happens, I have developed one algorithmic, low latency trading system in Common Lisp / ANSI C, and then was asked to rewrite it in Java which I did.

It actually traded on Warsaw Stock Exchange and was certified by WSE and was connected directly to it (no intervening software).

Yes, it is possible to do really low latency in Java. My experience is my optimized Java code is about 20-30% slower than what very optimized C code which is absolutely awesome. We are talking real research project funded by one of larger brokerage houses and the application was expected to respond to market signals within single digit microseconds every single time.

The issue with Java isn't that you can't write low latency code. The issue is that you are left almost completely without tools as you can't use almost anything from Java standard library and left scratching your head how to solve even simplest problem like managing memory without something coming and suddenly creating a lot of latency where you don't want it to happen.

Can't use interfaces, can't use collections, can't use exceptions, etc.

You can forget about freely allocating objects, except for very short lived objects you must relegate yourself to managing preallocated pools of objects.

I don't know what is the current state of garbage collection but in our case garbage collection had to be turned off and nothing outside of Eden could be collected during program execution (Java 8).

Writing that kind of optimized code is all about control and Java is all about abstracting it so you don't have to worry. As you see, both goals are at odds.

So I still prefer working low latency in C as it is more natural, native solution for managing problem where you absolutely need to control memory layouts, preallocated object pools, NUMA, compilation of decision trees to machine code, prefetcher, etc.

I have about 10 years combined experience working with C and 15 years working with Java.

Re:Java is better than C++ for high speed trading systems by rafaelvasco on Thursday November 26, 2020 @01:53PM (#25222636)
Yeah. Languages like C# and Java offer you so much facilities and QOL features compared to C and C++, but in practice you have to give up most of those facilities in order to have it perform almost on par with lower level languages; Even then I think it's still acceptable since you still have the tooling, the facilitated debugging, etc. C# for example, is keeping up with this trend of using higher level languages to code things closer to the metal and with more memory management awareness, and offering more and more features to make that kind of code more performant and with less heap allocations, since C# 7.0 or so: Span<T> for example is a revolution; But I use things like stackalloc, ref, fixed, Unsafe.CopyBlockUnaligned, &MemoryMarshal.GetReference etc, on a daily basis to get that extra performance where it matters;
Re:Java is better than C++ for high speed trading systems by pjmlp on Thursday November 26, 2020 @05:01PM (#25223831)
That was exactly what made me pick C++ over C already in 1992, ability to code at all levels in a modern programming language.

Nowadays I rather use Java and .NET, which have been picking up on the features that should have been there at v1.0, like the ones you mentioned, but hey better later than never. :)

And on the few cases where it isn't enough, I can code a tiny library in C or C++, no need to throw everything away.

Re:Java is better than C++ for high speed trading systems by DC-3 on Thursday November 26, 2020 @02:33PM (#25222912)
So Java is nearly as low-latency as C if you're willing to write your Java as if it were C?
Re:Java is better than C++ for high speed trading systems by lmilcin on Thursday November 26, 2020 @02:45PM (#25222991)
It can be done. It is just not very practical, you are better off writing C.

The only situation I can think of where Java would be right fit is where you have an application with a lot of logic and just a small part that needs to be highly optimized.

Re:Java is better than C++ for high speed trading systems by peter_lawrey on Monday November 30, 2020 @06:12AM (#25252634)
Most trading systems I work on only need a very small portion to be highly optimised. As long as most of the code is pretty efficient that is fine.
Re:Java is better than C++ for high speed trading systems by kgwgk on Friday November 27, 2020 @03:51AM (#25226709)
Real programmers can write FORTRAN in any language.
Re:Java is better than C++ for high speed trading systems by iphorde on Thursday November 26, 2020 @03:56PM (#25223468)
No. Java is not low-latency by any stretch. Java advocates would like to think it is, until a Bank, like BoA spends $8 million on Java in 2014, just to have it fail compliance testing. Entire project was scrapped.
Re:Java is better than C++ for high speed trading systems by pjmlp on Thursday November 26, 2020 @05:03PM (#25223835)
Re:Java is better than C++ for high speed trading systems by iphorde on Friday November 27, 2020 @12:05AM (#25225683)
250ms is not real-time, nor high frequency.
Re:Java is better than C++ for high speed trading systems by lmilcin on Friday November 27, 2020 @06:50AM (#25227876)
Please, get informed on what real-time is before you comment (in general it is good idea to understand a concept before you place confident comments like yours).

https://en.wikipedia.org/wiki/Real-time_computing

Real-time has nothing to do with performance and everything to do with predictability and deadlines.

People are put off when they find out that real time versions of Java or Linux are actually slower than the regular ones. Until you think about it a little bit and if they were faster there would not be multiple versions.

Real-time versions of products are created because sometimes performance is achieved by means of stochastic processes or amortizing costs and you need to remove these optimizations to be able to support deadlines.

Real-time programming emphasizes ability to calculate how long an operation will take as more valuable than raw performance.

Re:Java is better than C++ for high speed trading systems by sam_bristow on Friday November 27, 2020 @03:35AM (#25226620)
250ms could easily be real time in the context of a given system.

Real time doesn't (necessarily) mean fast, it means the latency is bounded.

Re:Java is better than C++ for high speed trading systems by pjmlp on Friday November 27, 2020 @01:37AM (#25226061)
It might not be high frequency, but it certainly is real-time, unless you are implying that the agencies that give this kind of government certifications are all a bunch of fools.
Re:Java is better than C++ for high speed trading systems by eru on Friday November 27, 2020 @02:50AM (#25226403)
Doesn't real time just mean that you can guarantee to react within certain time limits? In principle, those time limits could be hours or days..
Re:Java is better than C++ for high speed trading systems by pjmlp on Friday November 27, 2020 @05:33AM (#25227398)
Yes, but those limits are part of certification processes like those done by companies like https://www.highintegritysystems.com.

Usually playing with numbers in these kind of deployments isn't part of the package.

Re:Java is better than C++ for high speed trading systems by peter_lawrey on Monday November 30, 2020 @06:16AM (#25252660)
The trading systems we work on have a wire-to-wire latency of 20 - 50 microseconds on the 99.9%ile. It really depends on what you are looking to achieve.
Re:Java is better than C++ for high speed trading systems by the_arun on Thursday November 26, 2020 @07:37PM (#25224595)
I think this may not due to programming language, but beyond that.
Re:Java is better than C++ for high speed trading systems by ByteJockey on Thursday November 26, 2020 @02:40PM (#25222949)
You're really just paying for the jvm overhead at that point. 20-30% seems in the right ballpark.
Re:Java is better than C++ for high speed trading systems by junon on Thursday November 26, 2020 @02:13PM (#25222782)
I'm so happy this is the top comment. I'm saving this as it's well written and highly qualified. Thanks for sharing.
Re:Java is better than C++ for high speed trading systems by saghm on Thursday November 26, 2020 @03:13PM (#25223191)
> So I still prefer working low latency in C as it is more natural, native solution for managing problem where you absolutely need to control memory layouts, preallocated object pools, NUMA, compilation of decision trees to machine code, prefetcher, etc.

Out of curiosity, what was the rationale for writing the system in Java? Were the tradeoffs not obvious until too late, was the choice of language dictated by higher-ups, etc.?

Re:Java is better than C++ for high speed trading systems by lmilcin on Thursday November 26, 2020 @03:35PM (#25223331)
The rationale was there were no developers available with skills to work on a complex Common Lisp application.
Re:Java is better than C++ for high speed trading systems by peter_lawrey on Monday November 30, 2020 @06:27AM (#25252720)
Projects with limit time and resources can be completed faster with a high level language giving you more time to optimise it. A well optimised Java application can perform better than a poorly optimised C++ application.
Re:Java is better than C++ for high speed trading systems by stingraycharles on Thursday November 26, 2020 @03:30PM (#25223310)
Probably the usual suspect, it’s easier to find engineers that know Java than C/C++.
Re:Java is better than C++ for high speed trading systems by RobLach on Friday November 27, 2020 @02:55AM (#25226423)
And it’s a foolish assessment. Finding Java programmers who can navigate all the hoops of low latency Java is more difficult than finding a C programmer who can do the same with C.
Re:Java is better than C++ for high speed trading systems by lmilcin on Friday November 27, 2020 @10:32AM (#25229422)
There is couple of reasons for this.

If you choose Java for your project it usually means you don't care a lot about latency and performance (you care some, but not all that much compared to, for example, linux kernel development). You care more about managing larger codebase, having easy access to whole lot of frameworks and libraries, good development tools and easy development / deployment process.

Developers who have experience with Java are inevitably people who have mostly had experience with those kinds of projects of which I would say most are corporate backend systems or Android applications. It really is difficult to find any other projects.

On the other C is inflicted pain of having much harder development environment for the size of the project. You only use low level language like C when you have a specific need. Maybe you need to develop Set Top Box software, or video codec, or some device controller, or high performance quant library, that kind of stuff.

So it isn't really fault of Java developers to not know the stuff. It is natural result of Java being suited to a different type of projects than C is.

Re:Java is better than C++ for high speed trading systems by news_to_me on Thursday November 26, 2020 @01:28PM (#25222399)
I'm curious, what role did Common Lisp play in the original version of your trading system?
Re:Java is better than C++ for high speed trading systems by lmilcin on Thursday November 26, 2020 @01:34PM (#25222463)
Common Lisp was doing all the "high level" stuff. No CL code was on the actual critical path but it still was responsible for some critical things that would be difficult otherwise.

For example, I had piece of Common Lisp code optimize and compile decision trees to machine code. These were used, for example, to check whether the trade is allowed to go to market. This had to be very optimized as it was running in parallel to constructing the message and had to deliver verdict before the message was actually created to have a chance at actually stopping anything from reaching exchange. This was my solution to have order verification in essentially zero time (the cost was single branch instruction that would only be mispredicted if the verdict was to stop the order).

Another piece of Common Lisp generated machine code to parse XDP messages coming from exchange. The spec for these was in XML, Common Lisp macros created extremely optimized machine code that used absolute minimum instructions to parse out the information we were interested in without having to write all that code manually. I actually used concept from Practical Common Lisp book for this (http://www.gigamonkeys.com/book/practical-parsing-binary-fil...). Thanks, Peter Seibel.

The entire application was essentially a Common Lisp application that used FFI to call a library with optimized ANSI C code. Some Common Lisp code was running in parallel to constantly update things like decision trees that required regular recompiling based on market situation but most was really just to setup everything and give control to the C part.

The application did not talk to operating system (Linux) after it started (no syscalls at all) and used full kernel bypass to talk to NIC.

Re:Java is better than C++ for high speed trading systems by matheusmoreira on Thursday November 26, 2020 @10:51PM (#25225417)
> did not talk to operating system (Linux) after it started (no syscalls at all) and used full kernel bypass to talk to NIC

That's really interesting... How does it work? How can a user space program talk to the NIC directly without using system calls? Shared memory?

Re:Java is better than C++ for high speed trading systems by eru on Friday November 27, 2020 @02:52AM (#25226408)
Re:Java is better than C++ for high speed trading systems by clappski on Friday November 27, 2020 @05:08AM (#25227223)
A lot of NIC manufactures provide either some hooks into something like DPDK or simple LD_PRELOADable libraries that offload the network syscalls (socket, poll, recv, etc.) into a library that talks to the NIC directly using mmap'd IO I presume.
Re:Java is better than C++ for high speed trading systems by Rebelgecko on Thursday November 26, 2020 @03:44PM (#25223390)
One interesting approach I heard about for doing garbage collection in finance is to get computers with a shitload of RAM and just turn off GC. At the end of the trading day you can reclaim memory by killing the process. Not sure how common that strategy is.
Re:Java is better than C++ for high speed trading systems by lmilcin on Thursday November 26, 2020 @07:24PM (#25224525)
That is exactly what I did. It was much less elegant and performant, though, than my initial solution with Common Lisp and C which basically allocated buffers in memory and mapped them with huge pages so that memory access don't ever require switch to kernel for mapping.

We also had custom Intel CPUs that are auctioned individually. And cutthrough switches that stream the IP packet immediately when they receive the IP header, before they receive the payload. I have calculated that if the message from exchange was a beam of light it would be delayed by less than a meter.

Re:Java is better than C++ for high speed trading systems by KMag on Friday November 27, 2020 @02:23AM (#25226268)
> would be delayed by less than a meter

That's just latency to your NIC, right? 1 meter ~ 3 ns ~ 9 CPU cycles at 3 GHz, less than the integer pipeline depth last I checked.

Re:Java is better than C++ for high speed trading systems by lmilcin on Friday November 27, 2020 @12:17PM (#25230513)
Latency through switch (ie. how much the switch added compared to if the signal traveled unimpeded in vacuum).

Decade ago.

Re:Java is better than C++ for high speed trading systems by dralley on Thursday November 26, 2020 @11:09PM (#25225479)
If that's your memory management strategy, then what do you actually gain by using Java?

May as well just use C and never free your memory. Lots of memory management issues go away if you never free memory.

Re:Java is better than C++ for high speed trading systems by peter_lawrey on Monday November 30, 2020 @06:31AM (#25252744)
Not all your services/code needs to have the lowest latencies so you can have a team which just needs to be able to translate the business requirements and a couple of people to optimise the critical code.
Re:Java is better than C++ for high speed trading systems by peter_lawrey on Monday November 30, 2020 @06:29AM (#25252732)
If you have multiple processes each with 24 GB of Eden space and you can produce 1 GB/hour of garbage in each service and still only GC once a day.
Re:Java is better than C++ for high speed trading systems by gbronner on Thursday November 26, 2020 @03:45PM (#25223399)
It works until it doesn't and you lose a ton of money
Re:Java is better than C++ for high speed trading systems by vbezhenar on Thursday November 26, 2020 @02:42PM (#25222964)
Sounds like it should be possible to wrap it all into some kind of library and set of guidelines. On a positive side you probably could have much more confidence in Java code, because lots of boring and hard to debug memory errors just not possible (but may be C professionals don't do those errors?).
Re:Java is better than C++ for high speed trading systems by lmilcin on Thursday November 26, 2020 @03:43PM (#25223385)
The cost of environment not making it possible for you to make a mistake is always there, somewhere.

I think best hypothetical development environment would allow you to specify the tradeoff and choose whether you want strong typing, fully managed memory or ownership semantics.

Re:Java is better than C++ for high speed trading systems by bigbubba on Thursday November 26, 2020 @01:51PM (#25222627)
Was a specialized real-time JVM used for this? I know real time isn't necessarily low latency, but it sounds like both are desired properties in this sort of situation.
Re:Java is better than C++ for high speed trading systems by junon on Thursday November 26, 2020 @02:14PM (#25222790)
Realtime and low latency are two different properties. Realtime systems have timing concerns and can oftentimes be slower than non-realtime systems.
Re:Java is better than C++ for high speed trading systems by lmilcin on Thursday November 26, 2020 @02:42PM (#25222961)
This is not strictly true.

Realtime means processing of at least some of the events and tasks in your application must be completed before a set deadline. For example, coin acceptor that detects what kind of coin you dropped in has x milliseconds to figure out if the coin is to be accepted or not. If it can't do it within the time you get unhappy customer.

Realtime systems don't have to be very performant, but they don't have something like 99th percentile latency shooting up. All events must be processed within deadline.

Low latency systems are soft realtime PLUS performance.

In a low latency trading system you have to respond to market within a deadline to have a chance to do something useful but if you don't normally it doesn't mean lives are lost, metal is bent, etc.

Re:Java is better than C++ for high speed trading systems by iphorde on Thursday November 26, 2020 @03:57PM (#25223476)
Realtime requires low-latency. In trading, they require real-time, not low-latency.
Re:Java is better than C++ for high speed trading systems by peter_lawrey on Monday November 30, 2020 @08:02AM (#25253290)
Some people use Azul Zing but we typically use OpenJDK or Oracle JDK.