I wonder if that would make java more suitable to game development since pauses need to be <8ms, ideally.

I think a lack of value types also hurts Java significantly here. Though I believe they're also on the roadmap.

Yeah, implementation is under way for value types; the project is called Valhalla - https://en.wikipedia.org/wiki/Project_Valhalla_(Java_languag...

With generational GC, does this even matter? Generation 0 is pretty much a stack as far as performance and operation go.

Every object requires a header and a 8 byte pointer. This means even an empty object will cost you 16 or even more bytes. If you have perfect data locality and zero cache misses then you will still suffer from exhausting your bandwidth on useless data.

Then there is the fact that you still cannot actually control the allocation. The allocation pattern ABBAABABBA has worse performance on read accesses of all As or all Bs than AAAAABBBBB but with two arena allocators you can write your code to allocate memory according to the irregular pattern but still have continguous data as the end result.

Problem is memory locality, or lack thereof. Having to chase pointers around and not having things laid out in sequence doesn't play well with the CPU cache.

I'm not arguing against value types as I definitely want those too but it's not that simple nowadays - a moving garbage collector (like all modern server jvm's have) can do things for locality that non-moving systems (gc'd or not) can't - https://shipilev.net/jvm-anatomy-park/11-moving-gc-locality/

That only applies in a few specific cases. Basically, a linked list or an array of objects containing no further references. Otherwise, those internal references get immediately followed and significantly dilute the locality.

There have been garbage collectors designed to improve locality of predictable accesses, but they are not the norm.

When you embed a composite object into another composite object, in Java there always needs to be a pointer indirection between them. Even if GC and allocation is free, this is a massive performance loss. (For example, if you have something that would fit into 4 bytes, now you have to fit a 8-byte pointer, and the minimum created object is probably 16 bytes.)

Alternatively you can just flatten everything into a single object, but this loses out on abstraction.

Can't the compiler flatten objects into one another?

For example, if one class includes a reference to another, merge the fields and methods of both classes into one big mega-class, then see if there is any redundant code which can be removed for this new megaclass.

Not in general, because of object mutation, subtyping, and the fact that object fields are semantically references. First, what happens if you update the value to point to a subclass with more fields? They won't fit in the data structure (there are some other problems too, but that's the biggest one). If the classes have to be final and can't be modified through reflection, you'd probably be closer to a solution, but now any time you take a reference to a value in the inner class, you will be pointing into an object of the outer class, which means that GC has to be able to figure out that the internal pointer is keeping the object containing it alive. That's a pretty tricky thing to do, at least in a precise GC. Ultimately, this is not the sort of optimization that a compiler can easily prove safe in general; you could prove it was safe in specific instances (e.g. a local variable that never escapes), but in such cases Java can often allocate the whole thing on the stack anyway (or even in registers). If you want the optimization to work in any other cases, creating something semantically different (that doesn't allow interior references and/or mutability) is a better option.

The GC still has to move objects that survive, and this isn't deterministic. It's fine for a lot of code, but for inner loops when latency matters, you want better guarantees than that.

It doesn't matter nearly as much as people believe. Value objects at this point are a kind of red herring. The JVM does aggressive escape analysis and is also very good at capturing short-lived objects. In the big picture, pointer chasing and memory latency is kind of the last frontier of performance grumps. It really becomes an issue is with very large object arrays that must be iterated over quickly. In that case it's straightforwards to implement your own value-types using code-gen or use excellent libraries like Chronicle-Values [1].

[1] https://github.com/OpenHFT/Chronicle-Values

I'm not sure why you argue against giving the compiler the information it needs to do its job, and instead recommend these crazy codegen hacks, that even describe themselves as "poor-mans" solutions.

> I'm not sure why you argue against giving the compiler the information it needs to do its job

There's several different factors to consider. Value objects will be a valuable addition to the core JVM if only to standardize the many different approaches in place today. But even the standard value types will have strict limitations -- they will likely be immutable and it's not clear that you'll be able to easily obtain their memory address. That means many projects will still need things like code-gen. Code gen, and buffer-backed objects in general, isn't a hack at all it's a proven, battle-tested solution for many domains (finance, scientific sims, big data) that are manipulating several several gigabytes of objects in a performant manner.

Can you please elaborate?

It's just an object that lives on the stack, not the heap. In lots of languages when you create an object, it's pretty explicit where you're putting it. When Java was created, they wanted to get rid of that complexity, so primitives always go on the stack and objects always* go on the heap.

* I think Java has escape analysis, which means that if it can determine that an object you created doesn't leave its context, it will go on the stack and won't add work for the garbage collector.

It's more about the memory layout than where the objects live. For example, consider a Point class with (x, y) members.

If you have an array of these, each element in the array will be a reference/pointer to a separately allocated Point instance which can be anywhere in the heap. Accessing the array is expensive due to lack of locality (arbitrary memory access). Additionally, each instance has substantial object overhead (typically 8 or 16 bytes).

There are various ways to deal with this. One way is to have separate arrays for x and y. Another way is to use a byte array and use Unsafe to read/write the integers. This is terrible from a developer standpoint and is only acceptable if it is a small part of your application.

Value types solve this by giving you an efficient memory layout while still allowing you to code against them as normal. The slogan is "codes like a class, works like an int".

Value types also give you more control over cache locality. Not having to travel that extra pointer redirection can really add up depending on what you're doing.

There is at least half a magnitude in speed benefit from a cache-line-size/alignment tuned B-tree compared to e.g. a red/black tree or similar, if used to 'only' store pointer-size value types, and even more if the value type takes up even less space. A case would be e.g. a float/int pair ordered by floats and used as a priority queue or such. That's an 8-byte-large value type. Though, it might, for that case, need to be doubled up if there is a uniqueness constraint on the integers associated with the floats. LLC cache pressure can be very real. I highly recommend perf stat -dd and perf top -g -e uops_executed.stall_cycles as well as the -e cycle_activity.stalls_ldm_pending. The former counts how long stalls happen, including those where instructions like DIV just take long to execute, the second counts stall events, i.e. one stall per load/store, not weighted for how long this stall event actually stalled the core. I'm pretty sure the former ignores cases where just one hyperthread stalls, but I'm not as sure for the latter. On Haswell and newer, unless using branch taken/not-taken profiling, I recommend setting perf config --system call-graph.record-mode=lbr before using perf top -g and perf record -g, as -fomit-frame-pointer binaries cause issues otherwise.

Moreover, they can be used in composite objects without pointer chasing. Suppose class A is a 32-byte value type. Then

    class B {
        A a1;
        A a2;
    }

is one continuous 64-byte block of memory. Without value types, each A will be allocated separately on the heap, and B will be a pair of pointers.

Lack of value types makes it hard to get cache efficient memory layouts in Java.

Of course if/when a1 gets also referenced from outside while you may not need B any more that is then the things become more interesting.

Depends how they implement value types. They might simply say 'you can't have a reference to a value type'. You either embed it in your class, or you copy it's value.

You mention it in your footnote, but objects don't necessarily live on the heap - they can also live on the stack, or in registers, or in vector pipelines, or not at all.

Also an interesting aside is that objects are never 'allocated on the stack'. As in you won't find the same layout of memory on the stack as you do on the heap. Instead objects are turned into data edges in the compiler's graph. It's much more abstract than literally doing an alloca.

For further details on how objects are not exactly allocated on the stack: https://shipilev.net/jvm-anatomy-park/18-scalar-replacement/

> It's just an object that lives on the stack, not the heap.

That's not quite correct. A reference type might contain a value type field, in which case the value type will still live on the heap.

Conversely, a local variable's type might be a value type, and _still_ involve a heap allocation. For example, std::vector in C++ is a value type but the array storage is heap-allocated.

So it's really about semantics, and not memory layout. Accessing an lvalue with a value type produces a copied rvalue; with reference types you're only copying the reference itself.

> For example, std::vector in C++ is a value type but the array storage is heap-allocated.

This isn't correct. Firstly, C++ doesn't have value or reference types. Where any object is stored depends purely on how it is allocated. Either in the free store (dynamic) or automatic storage (the stack).

std::vector has a constant object size - typically the size of 3 pointers (but that depends on the implementation - the standard doesn't specify): the start of the memory (dynamically allocated for storage), end/capacity (typically either another pointer or a size_t of the current capacity), and back (typically either a pointer just past the last element used or a size_t + 1 of current size). It's storage is dynamically allocated, though, and for a std::vector<T> will have dynamically allocated sizeof(T) * capacity bytes (maybe more due to alignment issues).

std::array is constexpr safe and does not (directly) dynamically allocate memory - size has to be known at compile time (hence it's a template argument).

For instance, on my compiler (GCC 4.8.4), for this example program:

  #include <iostream>
  #include <vector>
  #include <array>

  int main()
  {
        std::vector<int> v_ten(0, 10);
        std::vector<int> v_twenty(0, 20);
        std::array<int, 10> a_ten{0};
        std::array<int, 20> a_twenty{0};

        std::cout << "sizeof(v_ten): " << sizeof(v_ten) << std::endl;
        std::cout << "sizeof(v_twenty): " << sizeof(v_twenty) << std::endl;
        std::cout << "sizeof(a_ten): " << sizeof(a_ten) << std::endl;
        std::cout << "sizeof(a_twenty): " << sizeof(a_twenty) << std::endl;

        return 0;
  }

produces this output:

  sizeof(v_ten): 24
  sizeof(v_twenty): 24
  sizeof(a_ten): 40
  sizeof(a_twenty): 80

edit: added comment about alignment.

The parent is correct. std::vector is a value type in the sense that everyone else is talking about... All types in C++ are, as their members are not inherently stored on the heap. C++ (and C, and Rust) has types that behave like what other languages call value types, even if there is not any other sort of type.

The mention of array in the parent was referring to the dynamic storage of a vector, not std::array.

When Java was created, they wanted to get rid of that complexity, so primitives always go on the stack and objects always go on the heap.*

This was inherited from Smalltalk. Everything was an "object reference," with tag bits and all, but SmallInteger was optimized by using the low 16 bits of the object reference pointer to store the SmallInteger. So you can think of Smalltalk as having SmallInteger as a "primitive" type with pass by value.

I worked for a company where we had a 100-150 microseconds order latency. We had to tune jvm to extremes. But it is very doable to optimize Java gc for low gc pause times.

With those latency requirements why did you use Java?

Plenty of people are writing low-latency trading applications in Java where the latency budget is under 100 microseconds. This has been the case for several years now.

It's by no means new or even especially difficult these days. The resulting code is far more maintainable and robust than C++ solutions.

I've had the opposite experience. For green-field applications, it's far easier to write the application in C++ than Java & meet the latency requirements. Java requires far too much tuning, where as C++, from the get go, you can generally just glance at the code and have a good idea of the latency.

We're currently fighting a Java app that in general has decent latency (10s of usecs), but has outliers of greater than a second when GC kicks in. We don't have that issue with the C++ components of our trading system.

It's always going to be subjective. If you have people experienced in gc tuning, it's pretty easy to weed those gc outliers out. I am currently doing that for one of the applications for the company I work for. For me, it would be far easier to write a low latency, high concurrency Java app than a c++ app. The last c++ app that I was asked to write works horribly and bleeds memory. But it was not an important app.

Oof. There's probably some compounding factors that make it more difficult, but usually getting GC pauses <500ms isn't that hard with current-gen collectors. Either being very careful with CMS, or just feeding G1 a lot of memory.

I’ve heard a pretty effective, lazy strategy for this scenario is to oversize the heap to avoid GCs through the trading day and to simply restart the application before the next day. Unsure if it’s viable for all trading platforms, but it makes sense for some I’m sure.

Depends on your order volume. Sure an oversized heap may work for low order volume, but if you're dealing with high volume, even hundreds of gigabytes of RAM isn't even for a trading day.

And yeah, we restart everyday (we only trade US equities, so plenty of downtime for a restart).

You would be surprised to know majority of trading companies/exchanges use Java for trading platform. So does majority of companies in finance sector. I have been working in this secor for over 10 years and have interviewed for many of them.

I would be surprised. I was under the impression that kdb+ holds the majority role in fintech.

If you used to tune JVM only, that might be interesting. Most often the Java code is what being tuned - to the point where there is no point to use Java - with (almost) no gc, off-the-heap memory allocations and all that jazz.

It's not either/or. Everything needs to be tuned to achieve the final latency requirements. This includes tuning java code to eliminate creating "garbage", immutable string objects mostly, unnecessary object creation, creating short lived objects compared to long lived etc. The other part right sizing jvm generational partitions, using right gc algo etc. Next part would be to find out the code path with lowest latency requirement and making it hot/jitting. Even next part would be to tune OS for things like thread pinning etc etc. You can go further and tune the network layer. Every little thing counts.

You can go even further and employ what's called mechanical sympathy (https://mechanical-sympathy.blogspot.com/) and write your code in a way that makes it easier for the processor to process faster.

I've never worked with application that use off-heap allocations or no-gc requirements.

This doesn't make much sense. Using object pools and and off-heap data structures doesn't warp the code that much, if at all. You still end up with pretty much all clean, simple Java code.

I've seen (10+ years old) low-latency Java apps where people avoid using any OO (ie everything is static methods)... and even then the code was still often easier to maintain and understand than equivalent C++ applications. Nobody does this any more anyways.

8ms? Where do you get that number from?

If your player has a 144Hz monitor then your pause time, rounded to the nearest millisecond, has to be 0ms.

Where do you get your number from? 1/(144 Hz) is 7ms. 1/(60 Hz) is 17ms.

I get my number from decades of professional game programming.

In a 7ms frame, you are spending most of that frame doing the work of rendering the actual frame (unless your game is so trivial that the GC is going to be easy / fast anyway). An additional millisecond is going to cause you to miss your deadline and drop a frame. Dropped frames feel really bad.

This sort of stalling is only caused by extremely short lived objects. Pooling and provisioning your data ahead of time goes a long way to fixing these specific issues to the degree that they become unnoticeable. There's a lot of tooling built into LWJGL to support this along with their own vector classes which are easily reclaimed.

Also has it really been a decade since Braid? Wow.

This is true, but it mainly just mitigates the problem. It can never solve the problem, because the whole concept of this kind of scheme is that the memory manager has some volition of its own ... thus it can choose to do things when you don't want or expect (actually this is unavoidable).

Also note that this category of answer is basically saying, "look, if you mostly manage your own memory, then GC takes less time!" That's true, but a large part of the value proposition of GC in the first place was to remove the burden of memory management. Once you are saying actually, GC won't do that for this class of application, then really what you are getting out of GC is memory safety (provided the rest of the language is memory-safe). On the one hand, hey, memory-safety is a benefit. On the other hand, I don't think very many people in game development would trade that much performance just for memory safety.

(And in fact in game development we very often have to do unsafe memory things. So really what ends up being said is "much of the system has memory safety" which, really, does not sound very alluring.)

> That's true, but a large part of the value proposition of GC in the first place was to remove the burden of memory management.

This is a good point. I've spent a fair amount of time trying to build performant stuff on the web, and my conclusion has been that if you really care about memory management, garbage collection is not your friend.

You spend more time than is necessary trying to convince the engine not to allocate anything you don't want, trying to lay out objects ahead of time, etc... I have spent more time learning about how Chrome's garbage collection works than I would have ever spent manually tracking references myself.

And the end result is that I still have to manually track references and be careful about where things are allocated - the garbage collector for me is objectively a net negative, not just for performance but for dev time and program complexity. And I'm not even really doing anything all that complicated in any of the software I build -- it is not hard to run into these kinds of issues.

This is something that I didn't understand until I saw it in my own software, but I strongly suspect that if I could get rid of browser garbage collection I would spend less time thinking about memory than I do now.

Since you're talking about the Chrome GC I'm going to assume that you're writing code in JS. If that is the case and you have to need such performance that you're having to worry about GC and tacking references then you are probably using the wrong language. Admittedly Web Assembly is pretty new but it sounds like that is your solution, or even asm.js would be worth considering.

> I have spent more time learning about how Chrome's garbage collection works than I would have ever spent manually tracking references myself.

Do you have any good references you can share regarding Chrome's garbage collection? This is a topic I would like to learn more about.

Unfortunately not - there are probably good resources out there, but most of what I know has come out of manual profiling in Chrome, different experiments, and the occasional StackOverflow post that explains a weird behavior or two.

At some point in the future I will probably write a blog post about good techniques for taking back control of JS memory management from the browser, but it probably won't be for a while.

The short answer though is that browsers try to defer garbage collection until it looks like the page isn't doing much; and the problem is that games don't ever really stop doing stuff. This encourages the browser to put off garbage collection and handle it in chunks, which in turn leads to dropped frames because you're doing a lot of deallocation at once.

This is why if you profile a Chrome app that's doing a lot of allocations very quickly, your memory usage will kind of sawtooth all over the place. And you get around that by getting rid of as many allocations as possible.

IMO the lack of garbage collection in WASM is the most exciting thing about it, but different languages have different tradeoffs and WASM isn't appropriate for every application.

you've talked about that before, "we very often have to do unsafe memory things", and that a lot of things done in game dev end up violating a lot of principles that are taught in college, but to be honest I'm not sure. granted, I haven't worked directly in game dev before, but i've worked on some projects where runtime perf was the _only_ requirement... and I'm just not convinced that these issues couldn't be worked around using constructs like smart pointers, weak pointers, and just general design principles...

that said, the witness is one of my favorite games of all time, so I think its safe to say you know what you're doing

The graphics APIs we use are often just about managing raw buffers of memory -- often different areas of memory that have different performance characteristics (Onion Bus vs Garlic Bus, yo). You simply can't do that under constraints of memory safety.

But beyond that ... "smart pointers" and the like make your program slow, because they postulate that your data is a lot of small things allocated far away from each other on the heap. I spoke about this in more depth at my Barcelona talk earlier this year.

Although I would argue that in games, memory safety is the door to game cheats, bots and piracy.

Wait, why have a GC if you're going to work around it and end up with manual memory management anyways?

So that when you make a mistake, instead of random memory corruption, you get a leak or a slower GC cycle.

I nearly spit my drink out from that dudes response, because I just happened to peek at your username as I was reading.

It's a crazy place here on HN.

60hz displays, giving a frametime of ~16.667ms. 8 is around half that, giving you opportunity to drop a frame, but not much more that.

Hrm, I would think that if you were developing a game in a GC environment, where GC timing turned out to be a problem, you would get rid of dynamic memory allocation, and try to do everything much more like embedded system programming with fixed memory blocks...?

I suppose at the complexity of modern games that's simply not possible anymore...

You can do that, and people do. However a GC language doesn't normally give you the flexibility of getting a block of memory and doing whatever you want inside it, instead you use things like object pools.

You can allocate a block of memory with `ByteBuffer.allocate()` or, if you want it outside the JVM heap, `ByteBuffer.allocateDirect()`.

You lose all the object/memory management features, though there are some projects around to provide struct-like APIs: https://github.com/alaisi/nalloc

System.Runtime.InteropServices.Marshal in C#'s case.

Also `fixed` in C# [1]

Not used Span<T> and Memory<T> [2] in anger yet, but they also seem to be attacking the issue from the other end

[1] https://docs.microsoft.com/en-us/dotnet/csharp/language-refe...

[2] https://blogs.msdn.microsoft.com/mazhou/2018/03/25/c-7-serie...

I consider object pools to be well within that family of problem solving that I'm talking about.

For a subset of developers, definitely. Overall I don't think it would lead to significantly more adoption, as the main draw (write once deploy everywhere) is already accomplished by engines like Unity.

On smaller heaps it was already possible to get 5-10ms pauses with the (now deprecated) CMS collector. Unlike Z it was a non-compacting collector though and had some ugly worst-case behaviors.

Yeah, CMS is great until it isn't, and then it ruins your whole day. A bit of a black art to get tuned well, too, and when your workload changes slightly your tuning is no longer effective and the world stops for 30+ seconds. Hopefully ZGC will be more like G1 in terms of "feed it some extra memory and set a latency target" being all the tuning you need most of the time.

Java is already suitable for game development, specially with LibGDX.

Not everyone needs to try to write the next Crisis in it.

It certainly has higher performace than all those HTML 5 and Flash games.

The biggest problem is that jMonkey is the only RAD tooling available and no match against any of engines that make use of C#.

> Java is already suitable for game development, specially with LibGDX.

I think it is safe to assume that under "game development" most people understand rather high performance demands, not a solitaire clone (esp since the comment you responded to pointed out pause-times as crucial).

You can also make a 3d game of large scale, and just let it have hiccups occasionally and still make a billion dollars.

Of course those of us hosting servers for that particular game would have saved a ton of money and pain if they had not used Java, or used Java more cleverly.

The point is that they should have used Java more cleverly.

One can make a game in C++ and make it just as bad.

The lack of value types makes the code quite unwieldy to control memory layout vs C++, or C#, or Go, or Rust or anything

Again, not every game needs to be the next Crysis.

Allocating every 3d coordinate on the heap probably wasn't a good idea.

Wasn't Minecraft originally written in Java? Might still be.

Minecraft also has generally awful performance (although this is generally attributed to its being coded poorly, not java).

Yes, but Microsoft ported it to C++ as means to have a common code base across mobile devices, game consoles and desktop computers.

Yes. But I believe that Microsoft made an improved version written in C++ after the acquisition.

"improved" is a matter of opinion. The Java version has mod support and the editions are kept at feature parity.

I remember when they promised a modding api... Not even the C++ has resulted in one... The fact that Notch went with Java might have been a blessing in disguise because it made the reverse engineering process semi automated.

That is known AAA game development.