Quite a few of the top HFT firms use Java, almost exclusively.

The trick is to segment your critical path code (whatever is executing the actual trades and is highly susceptible to delays) from any business logic. Then you can focus on making critical path components fast - disable GC completely, keep object allocations way down, audit every line of code, etc. With such a setup you can even do better than typical C++ because you avoid virtually all object allocation/deallocation costs that C++ has. Java object allocation is dirt cheap, it just hurts when you GC, which you can basically avoid or schedule for out of hours. If you want better I would not bother with C++ at all personally, but use C / assembly or even look at FPGA type setups.

This sounds like simulating arena allocation in Java by disabling GC temporarily. Except it's worse than arena allocator, because it's still mixed up in a general heap and costs more at collection. Not sure I buy the argument that this is cheaper than C++. After all, in C++ you could just drop the whole arena instead, so you get cheap alloc + super-cheap free.

Collection cost in that model is irrelevant as it's only a split second and happens after hours.

Yes, you can emulate a young generation in C++ or even multiple young gens (i.e. arenas), obviously, as you can do anything in C++ that you can do in Java and vice-versa (java has the Unsafe class that allows you to do low level programming with pointer arithmetic etc). The question is not, can it be done, but rather what's easier and lower cost?

With the Java approach, the only developer overhead is making sure you don't do too much allocation on the hot path, then sizing your young gen to avoid collections (one command line flag), and doing a GC at night. Not nothing but not too hard either; after all, you can allocate if you really need to, and some allocations will be optimised out anyway.

The C++ equivalent would be to allocate as much as possible on the stack, and then make sure to do a restart every night to eliminate any heap fragmentation that is left. Not that different, until you screw up and try to delete something on the stack or forget to delete something that isn't. That's when the robustness the GC is giving you starts to pay off.

> The question is not, can it be done, but rather what's easier and lower cost?

I thought it was about speed.

> The C++ equivalent would be to allocate as much as possible on the stack, and then make sure to do a restart every night to eliminate any heap fragmentation that is left.

The C++ equivalent would be to do heap allocation and never free, then shut the process down which will return its memory to the OS. Heap fragmentation wouldn't matter here since you aren't freeing any memory.

Fundamentally if you can avoid freeing any memory all day then you should just allocate it up front an be done with it. Anything else is ignoring the more fundamental point that you are using a limited amount of memory and pretending that you aren't.

Runtime speed is, as noted, going to be similar, with the winner being determined by which compiler optimisations benefit your code the most.

Yes you could also pre-allocate, but that's often more awkward than just allocating at the time of need, and besides, Java can turn heap allocations into not only stack allocations, but fully scalarise them (i.e. break down an allocation into a set of local variables and then delete any that are unused, move them around, etc).

> Runtime speed is, as noted, going to be similar, with the winner being determined by which compiler optimizations benefit your code the most.

This could not possibly be further from the truth. Getting rid of heap allocations can speed something up by ~7x. Cache locality can speed something up by 50x. Compiler optimizations do very little compared to accessing memory in a predictable way.

Indeed! In Java or dynamic langauges you are forced to always use a reference. Which means you'll get cache misses pretty much everywhere.

> The C++ equivalent would be to allocate as much as possible on the stack, and then make sure to do a restart every night to eliminate any heap fragmentation that is left.

I think that's a very brute-force approach. You could instead use an actual allocator which behaves the way you want.

For example this is how arena can work without any restart/stack magic: https://github.com/cleeus/obstack/blob/master/arena_test.cpp...

Same happens in google's protobufs https://developers.google.com/protocol-buffers/docs/referenc...

There's no need (almost) for manually wiping heaps at system level if your language allows you to change the allocator at any point of the app.

This is very interesting - is there any sample code one can learn from to build something in this style of programming ? Especially the deployment - do you have a scheduled that triggers GC or do you simply kill and restart, etc.

Actually you can trick the JVM to run a Full GC. If you run:

    jmap -histo:live <PID>

it will show a memory map of all the live objects and since java needs to run a full GC cycle to do that it will actually run a Full GC.

> better than typical C++ because you avoid virtually all object allocation/deallocation costs that C++ has

So to have Java perform better than C++ you just have to avoid allocation in Java and leave it in in C++?

> If you want better I would not bother with C++ at all personally, but use C / assembly or even look at FPGA type setups.

There is nothing in C that gives it a speed advantage over C++. Basically you've discovered step 1 of 3 when optimizing software (step 0 is to profile) -

1. Minimize heap allocations 2. Rework memory access for cache locality 3. Use SIMD

1 and 2 can be done with C++. 3 really needs to be done with something like ISPC so that SIMD use isn't fragile and compiler dependent.

I was going to say for people who are going to lengths like kernel bypass networking to get these trades out faster I wouldn't be surprised if the key stuff is in assembly.

I would be. Manually written assembly is not necessary better than what a compiler would generate and with super scalar pipelined architectures, seemingly better assembly can turn out to execute in the same amount of time. The things that matter are algorithmic.

I think you assume compilers are doing more work than they actually are. Processor models in the compiler are generally fairly simplistic and even ICC can easily be beat by hand tuned assembly.

Where compilers shine is allowing broad process-wide optimization.

> Quite a few of the top HFT firms use Java, almost exclusively.

such as whom?

If you're disabling GC, why use Java at all?

That's the point of using Java (or any interpreted language), isn't it? Avoiding manual memory management?

If you're going to go through all that horrid FactoryFactory Factory = new FactoryFactory(); bullshit, might as well get the benefits of the GC.

And if you don't want the GC, why not write your code in something else?

> If you're going to go through all that horrid FactoryFactory Factory = new FactoryFactory();

I get the impression that your only exposure to java is through jokes and blogs from dynamic typing evangelists.

How would that show any connection to dynamic typing? I imagine that the GP is coming from the perspective of C based on the fact that they're criticizing the use of garbage collection.

You'd be incorrect.

There are no popular languages that both have no GC and are comfortable to work in. Rust is going to be one, though I feel it's not entirely comfortable yet.

Garbage collection is just one of the things that make languages like Java comfortable to work in, but certainly not the only thing. Disabling automatic garbage collection in Java or C# I feel is a very effective way of gaining most of their advantages while limiting the effects of the runtime on the variability of your execution time.

Java is not an interpreted language by the way. FactoryFactories are a choice, not necessarily forced by the language.

How is your statement not one by someone who would find the modification of any language considered comfortable to remove garbage collection and require explicit free operations to render it uncomfortable?

Also, Java is interpreted at the byte code level by default. The way the hotspot JIT works is by detecting hot areas of code and then doing JIT to be compiled.

What sense does it make to speak of a language as being interpreted or compiled? These are attributes of an implementation. The existence of GCJ, a native-code Java compiler that's been around for years, proves this.

It's true there are languages that are essentially uncompilable, and so have to be interpreted. Perl is the best example I can think of; there are constructs in the language that can't be parsed correctly without runtime context. Python, Ruby, and JavaScript are all difficult to do AoT compilation on as well, at least if you want to do much optimization, but I wouldn't agree that they can't be compiled. But Java is not as dynamic as these languages.

When talking about Java, you can mean the standard library, the virtual machine or the language. The term Java itself is ambiguous.

I don't think it must be absolutely so that a language without garbage collection is uncomfortable, as I said I feel Rust comes very close.

The only popular languages without a garbage collector are either from the 70s (C) or based on a language from the 70s (C++xx). Both have extreme discomforts when compared to modern languages (Ruby/Haskell/C#/Rust).

Also, technically, that would make Java bytecode an interpreted language, not Java itself ;)

edit: Ok, apparently according to wikipedia I am wrong and Java generally is considered to be an interpreted language, I had a slightly different definition of interpreted in mind.

Yeah, when C displaced Pascal in CS programs (mid to late 80s?), it felt like an uncomfortable step backward even then.

Why were we programming in (little more than) assembler as our main language???

Mid to late 90's in Portugal.

When I came to learn C, I was already quite an expert in Turbo Pascal, version 6.0 by then.

Comparing C to Turbo Pascal 6.0 in terms of features and safety just felt like "meh!".

Thankfully I got someone that around the same time gave me a copy of Turbo C++ 1.0 and I learned how to get some of the Turbo Pascal features back, while having the C portability and joined the C++ ranks for a few years.

Similar experience. I picked up a copy of TP 5.5 and finally had an easier way to build my "ADT"s (abstract data type library type code).

THEN, I got a copy of TC++ a few months or a year later. WTF?!?

The light came on when I read Scott Meyer's "Effective C++" book in 96 or so. "Effective" meant "not stepping on one of 50 or so common land mines". I realized what a bad joke C++ was, and decided never to go back to it.

C is useful as a substitute for assembler, but not most applications. Too bad we didn't get to see more of Smalltalk in school in the 80s. And maybe Scheme instead of just a little bit of Lisp.

C++ first came into existence in 1983. C was made in the 1970s and was the basis for C++, but modern C is rather different than the original C because it assimilated C++'s strong type system. As for popular languages without a garbage collector, here is a list off the top of my head:

* C

* C++

* FORTRAN

* Objective C

* Objective C++

* Pascal

* Swift

Making a new Turing complete language is ultimately just an exercise in how to do the same things differently rather than better unless you find a way to construct a language that can use hardware more effectively than the existing languages can. Languages that lack garbage collection provide no incentive to wipe the slate clean for easier garbage collection.

There is an enormous difference between expert level use of C and languages like it and beginner use. There are static analysis and dynamic analysis tools that have been built to assist with catching bugs such as misuse of memory or undefined behavior. C and several others that I listed above are flexible enough that you can implement design patterns that you would expect to see in more "advanced" languages and with structured programming, you can use them fairly easily once they are written. Functional programming is doable:

https://github.com/cioc/functionalC

Object oriented programming is also doable:

http://ooc-coding.sourceforge.net

Generic programming can also be done using a mix of macros and void pointers. A rather powerful design pattern that I have seen in C is a function that encapsulates iteration on some data and invokes a callback that takes an accumulator that had been passed to it with the callback. It is great for iterating over things that are not expected to always fit in system memory. The non-accumulator version of that pattern is used in the POSIX standard ftw C function. The acculumator version feels much like generic programming as the type of the accumulator is known only to the caller and callback. The iteration function has no clue about the acculumulator's type. The same goes for plenty of in-memory data structures implemented with void pointers like lists and trees where the memory describing each node is encapsulated inside the object.

There is definitely a greater learning curve to C and languages like it, but once you are faililiar with the right patterns/abstractions, such languages are a joy to use and the advantages of more "advanced" languages look more like trade-offs rather than be killer features.

Also, people who are familiar with such languages tend to program differently. At work, I have been asked to write some userspace code in Go. I wanted to make some directory traversal code use as few CPU resources as possible (which is a design goal), so I asked for tips on how to do system calls from Go and horrified at least one Go programmer in #go-nuts on freenode in the process. Using the syscalls directly enabled me to take advantage of SYS_gentdents64's d_type to avoid doing a stat to determine whether an entry is a directory or not, increase the buffer to read more directory entries per syscall fewer calls per directory and detect when the end of directory by checking when the buffer has 65535 bytes of free space remaining, which reduces the verse the invocations from 2 to 1. A programmer that does everything the way that garbage collected language authors recommend likely would have had a far less CPU efficient traversal. The superfluous stat syscalls alone would have increased the number of syscalls by at least 1 order of magnitude.

I wrote a patch to glibc last night that enables readdir() to skip the second getdents64 call on small directories and I plan to submit after I have what I consider to be the final version. That ought to accelerate GNU find. I might give the Go OS package similiar treatment, although doing fast directory tree traversal the way I am doing it (which is similiar to what GNU find does) requires that to go OS package provide type information from getdents. That is a non-portable BSD extension that Linux adopted and consequently is something that I would not expect Go to provide.

Comfortable is a relative term. There are people comfortable working in C or C++.

True, there's a relevant photograph here[1]. Although I think that even those when they fall asleep or otherwise let their guard down their programming experience can get real uncomfortable real quick.

1] https://en.wikipedia.org/wiki/Bed_of_nails#/media/File:RGS_1...

>Java is not an interpreted language by the way.

Java compiles to native machine code?

Most of the time yes. The JIT will generate machine code for almost all usual code paths after little time.

Yes, it all depends which JDK you make use of.

Apparently many seem to think the OpenJDK is the only option available.

That factory code you mentioned isn't _just_ a Java thing, but a very well-known and wonderful design pattern.

Wikipedia says it best: "[The] Factory pattern deals with the instantiation of object without exposing the instantiation logic."

Practical benefits:

- Can help rid of excessive switch and if/elseif statements and splits up your logic

- Eliminates new keyword to improve testability

As with most language jokes/prejudices, I think the hate comes from seeing the misguided, incorrect or unnecessary uses of that pattern.

Java's prominence in the corporate world means sturgeon's law is more apparent.

Never heard of that law. Brilliant.

If you give Java a try, it's got a lot more impact than Factory. Play around with the language a bit, there are things in the JVM and the language that really change the way you think about programming.

Sure. I now know:

* What a great feature function pointers / procedural types / other call-by-name/reference-mechanisms were. (J8 lamdas are used like function references, but are kind of heavy)

* What a source of mystery "annotations" are.

Java started out like an interpreted version of Delphi or UCSD Pascal disguised as a cleaned up C++, but since 1.1 it evolved slowly and/or in the wrong ways.

Actually, as somebody who has been programming since 1983, in quite a few languages, Java brought little to nothing new to the table. Nothing personal against you, I'm just sick of the Java language/syntax and how it failed to live up to the hype.

I'm a fan of Java in some senses, but curious to here what things you discovered in the language or JVM that changed how you thought about programming

For me it is the vastly superior tooling.

* Best IDEs around. (Yes better than .net ecosystem)

* JVM monitoring (Flight Control, VisualVM)

* Performance Sampling/Profiling <1% perf cost

* actually being cross platform (yes, mono, bla)

Ability to use Clojure, Scala, Freje, Groovy, Java, Javascript, Python etc all on the one platform and share libraries and objects between each other.

People forget that Java has the best ecosystem of libraries bar none especially when you get into the more enterprise areas e.g. Big Data.

One-and-done is my favorite way of thinking about using Java. The JVM provides the perfect, for my use case, amount of abstraction from the hardware while still allowing you to do some amazing things.

There is also a huge amount of features that are supported by the JVM that Java still doesn't make use of.

There is also a great JIT and garbage collector.

Yes, but I suppose this is more the JVM than the language itself. The JVM is quite convincing though, hence Clojure, Scala, etc.

Because Java is the new COBOL -- especially if you statically allocate everything for performance reasons. At that point, you really are left with something not unlike COBOL with separate compilation and the ability to allocate scratch counters (or other primitives) on the stack locally, but all of your main data is still copy books^H^H^H^H^H^H^H beans allocated in the DATA DIVISION.

My experience with trading systems led largely to the same conclusions.

You CAN write GC code that does very little collection, but you end up being allocation sensitive anyway, so you may as well write c++ code.

For instance I wrote some .NET where instead of logging with strings, I'd use StringBuilder and try to get some reuse. Problem is SB wasn't written for that, and will allocate again if your new string gets too big. In the end it ended up being a pool of stringbuilders of various lengths. C++ would not have had this issue.

The problem with GC tends to be something alluded to in the article. When trading is brisk, a lot of data is coming in. So memory pressure is higher, leading to more GC. That's exactly when you don't want it. And debugging managed code I always find is more complicated than debugging C++. You need a whole suite of memory debuggers, all with their own idioms, and then it only really gives you a hint. With C++, you can valgrind et al where you can override new() and put in some accounting code. And it's normally quite clear where you allocated.

You can also just tell the OS you'll deal with memory yourself. Just get a big piece, and keep track of what's what using various arenas and such constructs. That way you're not even exposed to the variance in the OS allocation time (time to find space for a new object).

I'm not saying GC is a great idea but c++ doesn't necessarily solve your allocation problem: https://groups.google.com/a/chromium.org/forum/m/#!msg/chrom...

It seems like high performance systems should just be coded like microcontrollers, no OS or minimal embedded real time OS and you manage all your memory yourself.

Yes that's right. Essentially you need complete control over what happens, without having to pass control back to the OS.

Of course just using c++ is not a solution. There's a bunch of guidelines you need to stick to, especially with string.

What I find sad is that Java garbage collection isn't better than manual memory management, given the vast investment in the jvm. There are plenty of ways in which garbage collection can be faster, but production software is still in the stone ages when compared to research from 10 years ago.

For example, any scenario where Rust doesn't even need a lifetime annotation is a scenario where a garbage collecting language compiler still could manually manage collection at almost zero cost. It could, very simply I might add, make inferences of the most efficient way to collect block scoped objects...the vast majority of which end up getting collected in the short lifetime pool. Long lifetime objects might be more efficiently reference counted, so why can't the compiler decide that singletons should be reference counted, or even statically allocated instead of pinged during every large collection? And why can't we have an actual disposable interface that actually performs a manual collection?

For a language that is as memory heavy as Java is, with as much boxing that it does, and the high performance demands of its user base, it blows my mind that their best idea for improvement in garbage collection in the last decade was to naively go after more concurrency. I realize that is what everybody thought the future was for everything, but something something Amadahl's law. There are so many better opportunities that involve allocating less, destroying without a mark stage, etc.

What you're describing is escape analysis and the resulting SROA. These are things that the HotSpot VM does in fact do.

That is news to me (thanks for the clarification), but unless I'm misunderstanding you, escape analysis is only for block scoped allocations. It doesn't perform any optimizations for large and/or long lived objects so they can bypass expensive large heap mark stages, and it doesn't allow for deterministic destructor interfaces. As far as I can tell, the only industrial GC language out there attempting to do this sort of thing is Swift.

Deterministic destruction RAII style is handled in GCd languages with try-with-resources or using type constructs.

Modern collectors like G1 don't do full mark/sweeps of the entire heap in normal operation - only in rare cases like if you request a memory profile, or if the entire process runs out of RAM and needs to do a hard stop. And reference counting isn't something you can just automatically do due to the presence of cycles.

Notwithstanding these things, you may be interested in Graal, and specifically this talk on automatic region allocation:

https://www.youtube.com/watch?v=3JphI1Z0MTk&list=PLX8CzqL3Ar...

http://www.oracle.com/technetwork/java/jvmls2015-wimmer-2637...

A program that segfaults at the wrong moment can ruin your day. It is a tradeoff. It boils down that bad memory handling produces bad results. In C++ these are memory leaks or dereferencing invalid pointers, in GC languages, you pay with performance impacts. And, very occasionally one technology is completely unsuited for a given task. But that should not lead to such blanket statements.

Well Java can crash and slow down due to GC pauses. But at least you get a nice stack trace.

You could call Java a very versatile language :)

This sounds like a hohum IT department.

Finance is software and has been for a while. Competent players in this field wouldn't base entire implementation strategy based on one expert view. More likely, they would have more substantial trade off discussions.

I would this shop had much bigger problems than choice of stack.

There's a difference between using a tool that's not up to the job and employing people who aren't up to the job. Java is just fine for this kind of application.

Did he try Azul's Zing or their previous Vega hardware solution for pauseless JVM GC?

Not free by any means in $$$ or performance, but they should have avoided any pauses due to GC.

Do you actually mean that I need an expensive, dedicated hardware accelerator to get Java's performance to match C++?

Why not do as they actually did: use C++ ? Sounds simple enough... and much cheaper.

No, in many cases the JVM performance will match and sometimes even outperform a C++ implementation in terms of performance and I've seen a few examples of that, although math-heavy operations is usually not one of them. In terms of size and memory usage, a C++ implementation will almost always win.

The problem with Java is that it's not as deterministic in terms of performance as C++, so in areas where that is a must, it has to be taken into account. Even after saying that, the GC can, and indeed should, be tweaked and optimized. The JVM gives you more tweaking options and tools to optimize the garbage collection than any other platform I've worked with, but most developers don't know about them or don't care.

In cases like the original comment mentioned, it may have been better to continue using the C++ implementation, or it might have been useful to spend a bit less in consultants and developers and a bit on Azul's Zing (and maybe a few people that understood the platform). It might have saved a lot in the long term and indeed there are plenty of algorithmic trading solutions written in Java.

A few things to consider:

* it tends to be harder to write good C++ code than Java. For example we think that an average person can write better java code than C++ code because java is simpler to get going with. We also think that the average developer makes fewer mistakes in java than C++ (because C++ allows you to do 'crazy things!')

* that means that you need (on average) a better developer for C++, and that tends to cost more

* java tooling is exceptional (probably the best there is?)

* making small changes and releasing is much quicker in Java; builds tend to be shorter in duration for example

* once you have the framework for your system, the differentiator is then the business benefits delivered. If you can push small changes out very quickly you're getting rewards quicker and you can experiment more

* once you get down to certain performance points, it becomes similar writing java code as writing C++ code, but the JVM is also doing various work that you can't control so much. That can then inhibit you.

Other things:

* Azul isn't just h/w development it's also now in kernel plugin form iirc

* Have a look at Aeron (https://github.com/real-logic/Aeron) and I think the C++ and Java versions are fairly similar performance and they're designed by super with decent (industry renowned) developers writing it

* A lot of people write high performance code making extensive use of templates. That's a totally different way of programming than usual C++

Boring C++ is often faster than boring Java code. It's safer to write hacky Java, but code reviews should not let hacky code in.

C++ costs more, but that's often a non issue for a trading floor. Security is much simpler in the Java world, but also possible in boring C++ code.

Making small changes is really more a question of your code than anything else IMO. There is a lot of crazy C++ and Java code out there.

Now for basic CRUD apps Java is a clear win. For high preformance trading that's an open issue. But, IMO there are much better languages to use.

What about a third way? I have not used the GNU version of Ada or Pascal, but you get C/++ style allocation choices (static, stack, heap) without all of the safeties turned off.

Having bounds checks (with line number error reporting, rather than secondary, tertiary... damage followed by a core dump), and the use of non-null references (as well as pointers for the initial allocation from the heap) goes a long way towards eliminating much of the time wasting bullshit that C/++ brings into your life.

C is good for portable assembler. Otherwise, I want something less wacky than C++ to be "effective" with at a medium low level, and something higher level than Java for most other apps.

Where are you going to get a team of Ada/Pascal developers from ?

And tools to support them ?

And Enterprise Management has spoken!

Ignoring the fantasy that I would get to use something higher level (than Java) for most work...

Not everybody panics if they have to write code outside of Eclipse or Visual Studio.

Once upon a time, there was an expectation that programmers knew, or could learn, more than one language.

Sorry about the rude response, but I'm really sick of the dumbing down of everything for (counterproductive) business or "risk management" purposes. To paraphrase, "average performance" in this industry is pretty poor.

Azul isn't THAT expensive. These days it's just x86 software anyway. Financial trading firms are actually one of their top customers, if I understand correctly, and Oracle is developing features for low latency work partly driven by the demands of algorithmic trading shops.

OK, but it's more expensive that "free", and they didn't previously list their prices, but now I see at the excellent https://www.azul.com/products/zing/zinqfaq/

Zing is priced on a subscription basis per server.... The annualized subscription price for Zing per physical server ranges from $3500 (for several hundred servers) to $8000. Higher volumes and longer subscription terms will reduce the per-server price for Zing. Pricing for virtual servers is also available upon request.

Since you're likely to be running it on a server with a hardware cost approximately the same or larger, next time I'll only use two dollar signs ($$) ^_^.

And I've indeed heard it's popular with financial trading firms for the obvious reasons. Up to 1 TiB heap, no problem, and no 1,024 second pause which other collectors require for a full GC.

C++ is just free, because you happen to have clang and gcc to choose from among a pool of otherwise commercial compilers.

I remember when all C++ compilers worth using were commercial.