News for hackers. Stuff that matters.
Re:Java’s new garbage collector promises low pause times on ... by nicktelford on Thursday August 30, 2018 @11:54AM (#17877707)
I think a lack of value types also hurts Java significantly here. Though I believe they're also on the roadmap.
Re:Java’s new garbage collector promises low pause times on ... by jakewins on Thursday August 30, 2018 @07:42PM (#17881795)
Yeah, implementation is under way for value types; the project is called Valhalla - https://en.wikipedia.org/wiki/Project_Valhalla_(Java_languag...
Re:Java’s new garbage collector promises low pause times on ... by pkulak on Thursday August 30, 2018 @01:21PM (#17878613)
With generational GC, does this even matter? Generation 0 is pretty much a stack as far as performance and operation go.
Re:Java’s new garbage collector promises low pause times on ... by imtringued on Thursday August 30, 2018 @04:43PM (#17880565)
Every object requires a header and a 8 byte pointer. This means even an empty object will cost you 16 or even more bytes. If you have perfect data locality and zero cache misses then you will still suffer from exhausting your bandwidth on useless data.

Then there is the fact that you still cannot actually control the allocation. The allocation pattern ABBAABABBA has worse performance on read accesses of all As or all Bs than AAAAABBBBB but with two arena allocators you can write your code to allocate memory according to the irregular pattern but still have continguous data as the end result.

Re:Java’s new garbage collector promises low pause times on ... by pdpi on Thursday August 30, 2018 @04:13PM (#17880309)
Problem is memory locality, or lack thereof. Having to chase pointers around and not having things laid out in sequence doesn't play well with the CPU cache.
Re:Java’s new garbage collector promises low pause times on ... by wrmsr on Thursday August 30, 2018 @05:54PM (#17881139)
I'm not arguing against value types as I definitely want those too but it's not that simple nowadays - a moving garbage collector (like all modern server jvm's have) can do things for locality that non-moving systems (gc'd or not) can't - https://shipilev.net/jvm-anatomy-park/11-moving-gc-locality/
Re:Java’s new garbage collector promises low pause times on ... by ghusbands on Saturday September 01, 2018 @02:30AM (#17890560)
That only applies in a few specific cases. Basically, a linked list or an array of objects containing no further references. Otherwise, those internal references get immediately followed and significantly dilute the locality.

There have been garbage collectors designed to improve locality of predictable accesses, but they are not the norm.

Re:Java’s new garbage collector promises low pause times on ... by Tuna-Fish on Thursday August 30, 2018 @02:18PM (#17879247)
When you embed a composite object into another composite object, in Java there always needs to be a pointer indirection between them. Even if GC and allocation is free, this is a massive performance loss. (For example, if you have something that would fit into 4 bytes, now you have to fit a 8-byte pointer, and the minimum created object is probably 16 bytes.)

Alternatively you can just flatten everything into a single object, but this loses out on abstraction.

Re:Java’s new garbage collector promises low pause times on ... by londons_explore on Thursday August 30, 2018 @05:07PM (#17880774)
Can't the compiler flatten objects into one another?

For example, if one class includes a reference to another, merge the fields and methods of both classes into one big mega-class, then see if there is any redundant code which can be removed for this new megaclass.

Re:Java’s new garbage collector promises low pause times on ... by Jweb_Guru on Thursday August 30, 2018 @05:19PM (#17880872)
Not in general, because of object mutation, subtyping, and the fact that object fields are semantically references. First, what happens if you update the value to point to a subclass with more fields? They won't fit in the data structure (there are some other problems too, but that's the biggest one). If the classes have to be final and can't be modified through reflection, you'd probably be closer to a solution, but now any time you take a reference to a value in the inner class, you will be pointing into an object of the outer class, which means that GC has to be able to figure out that the internal pointer is keeping the object containing it alive. That's a pretty tricky thing to do, at least in a precise GC. Ultimately, this is not the sort of optimization that a compiler can easily prove safe in general; you could prove it was safe in specific instances (e.g. a local variable that never escapes), but in such cases Java can often allocate the whole thing on the stack anyway (or even in registers). If you want the optimization to work in any other cases, creating something semantically different (that doesn't allow interior references and/or mutability) is a better option.
Re:Java’s new garbage collector promises low pause times on ... by skybrian on Thursday August 30, 2018 @01:34PM (#17878762)
The GC still has to move objects that survive, and this isn't deterministic. It's fine for a lot of code, but for inner loops when latency matters, you want better guarantees than that.
Re:Java’s new garbage collector promises low pause times on ... by dnomad on Thursday August 30, 2018 @02:14PM (#17879188)
It doesn't matter nearly as much as people believe. Value objects at this point are a kind of red herring. The JVM does aggressive escape analysis and is also very good at capturing short-lived objects. In the big picture, pointer chasing and memory latency is kind of the last frontier of performance grumps. It really becomes an issue is with very large object arrays that must be iterated over quickly. In that case it's straightforwards to implement your own value-types using code-gen or use excellent libraries like Chronicle-Values [1].

[1] https://github.com/OpenHFT/Chronicle-Values

Re:Java’s new garbage collector promises low pause times on ... by arcticbull on Thursday August 30, 2018 @04:58PM (#17880692)
I'm not sure why you argue against giving the compiler the information it needs to do its job, and instead recommend these crazy codegen hacks, that even describe themselves as "poor-mans" solutions.
Re:Java’s new garbage collector promises low pause times on ... by dnomad on Thursday August 30, 2018 @05:18PM (#17880866)
> I'm not sure why you argue against giving the compiler the information it needs to do its job

There's several different factors to consider. Value objects will be a valuable addition to the core JVM if only to standardize the many different approaches in place today. But even the standard value types will have strict limitations -- they will likely be immutable and it's not clear that you'll be able to easily obtain their memory address. That means many projects will still need things like code-gen. Code gen, and buffer-backed objects in general, isn't a hack at all it's a proven, battle-tested solution for many domains (finance, scientific sims, big data) that are manipulating several several gigabytes of objects in a performant manner.

Re:Java’s new garbage collector promises low pause times on ... by shanxS on Thursday August 30, 2018 @01:10PM (#17878487)
Can you please elaborate?
Re:Java’s new garbage collector promises low pause times on ... by pkulak on Thursday August 30, 2018 @01:20PM (#17878590)
It's just an object that lives on the stack, not the heap. In lots of languages when you create an object, it's pretty explicit where you're putting it. When Java was created, they wanted to get rid of that complexity, so primitives always go on the stack and objects always* go on the heap.

* I think Java has escape analysis, which means that if it can determine that an object you created doesn't leave its context, it will go on the stack and won't add work for the garbage collector.

Re:Java’s new garbage collector promises low pause times on ... by electrum on Thursday August 30, 2018 @01:59PM (#17879028)
It's more about the memory layout than where the objects live. For example, consider a Point class with (x, y) members.

If you have an array of these, each element in the array will be a reference/pointer to a separately allocated Point instance which can be anywhere in the heap. Accessing the array is expensive due to lack of locality (arbitrary memory access). Additionally, each instance has substantial object overhead (typically 8 or 16 bytes).

There are various ways to deal with this. One way is to have separate arrays for x and y. Another way is to use a byte array and use Unsafe to read/write the integers. This is terrible from a developer standpoint and is only acceptable if it is a small part of your application.

Value types solve this by giving you an efficient memory layout while still allowing you to code against them as normal. The slogan is "codes like a class, works like an int".

Re:Java’s new garbage collector promises low pause times on ... by hermitdev on Thursday August 30, 2018 @04:03PM (#17880225)
Value types also give you more control over cache locality. Not having to travel that extra pointer redirection can really add up depending on what you're doing.
Re:Java’s new garbage collector promises low pause times on ... by namibj on Thursday August 30, 2018 @09:13PM (#17882189)
There is at least half a magnitude in speed benefit from a cache-line-size/alignment tuned B-tree compared to e.g. a red/black tree or similar, if used to 'only' store pointer-size value types, and even more if the value type takes up even less space. A case would be e.g. a float/int pair ordered by floats and used as a priority queue or such. That's an 8-byte-large value type. Though, it might, for that case, need to be doubled up if there is a uniqueness constraint on the integers associated with the floats. LLC cache pressure can be very real. I highly recommend perf stat -dd and perf top -g -e uops_executed.stall_cycles as well as the -e cycle_activity.stalls_ldm_pending. The former counts how long stalls happen, including those where instructions like DIV just take long to execute, the second counts stall events, i.e. one stall per load/store, not weighted for how long this stall event actually stalled the core. I'm pretty sure the former ignores cases where just one hyperthread stalls, but I'm not as sure for the latter. On Haswell and newer, unless using branch taken/not-taken profiling, I recommend setting perf config --system call-graph.record-mode=lbr before using perf top -g and perf record -g, as -fomit-frame-pointer binaries cause issues otherwise.
Re:Java’s new garbage collector promises low pause times on ... by blt on Thursday August 30, 2018 @02:01PM (#17879045)
Moreover, they can be used in composite objects without pointer chasing. Suppose class A is a 32-byte value type. Then

    class B {
        A a1;
        A a2;
    }
is one continuous 64-byte block of memory. Without value types, each A will be allocated separately on the heap, and B will be a pair of pointers.

Lack of value types makes it hard to get cache efficient memory layouts in Java.

Re:Java’s new garbage collector promises low pause times on ... by trhway on Thursday August 30, 2018 @03:48PM (#17880106)
Of course if/when a1 gets also referenced from outside while you may not need B any more that is then the things become more interesting.
Re:Java’s new garbage collector promises low pause times on ... by londons_explore on Thursday August 30, 2018 @05:09PM (#17880804)
Depends how they implement value types. They might simply say 'you can't have a reference to a value type'. You either embed it in your class, or you copy it's value.
Re:Java’s new garbage collector promises low pause times on ... by chrisseaton on Thursday August 30, 2018 @01:26PM (#17878670)
You mention it in your footnote, but objects don't necessarily live on the heap - they can also live on the stack, or in registers, or in vector pipelines, or not at all.

Also an interesting aside is that objects are never 'allocated on the stack'. As in you won't find the same layout of memory on the stack as you do on the heap. Instead objects are turned into data edges in the compiler's graph. It's much more abstract than literally doing an alloca.

Re:Java’s new garbage collector promises low pause times on ... by pgwhalen on Thursday August 30, 2018 @06:23PM (#17881325)
For further details on how objects are not exactly allocated on the stack: https://shipilev.net/jvm-anatomy-park/18-scalar-replacement/
Re:Java’s new garbage collector promises low pause times on ... by slavapestov on Thursday August 30, 2018 @02:22PM (#17879296)
> It's just an object that lives on the stack, not the heap.

That's not quite correct. A reference type might contain a value type field, in which case the value type will still live on the heap.

Conversely, a local variable's type might be a value type, and _still_ involve a heap allocation. For example, std::vector in C++ is a value type but the array storage is heap-allocated.

So it's really about semantics, and not memory layout. Accessing an lvalue with a value type produces a copied rvalue; with reference types you're only copying the reference itself.

Re:Java’s new garbage collector promises low pause times on ... by hermitdev on Thursday August 30, 2018 @04:24PM (#17880406)
> For example, std::vector in C++ is a value type but the array storage is heap-allocated.

This isn't correct. Firstly, C++ doesn't have value or reference types. Where any object is stored depends purely on how it is allocated. Either in the free store (dynamic) or automatic storage (the stack).

std::vector has a constant object size - typically the size of 3 pointers (but that depends on the implementation - the standard doesn't specify): the start of the memory (dynamically allocated for storage), end/capacity (typically either another pointer or a size_t of the current capacity), and back (typically either a pointer just past the last element used or a size_t + 1 of current size). It's storage is dynamically allocated, though, and for a std::vector<T> will have dynamically allocated sizeof(T) * capacity bytes (maybe more due to alignment issues).

std::array is constexpr safe and does not (directly) dynamically allocate memory - size has to be known at compile time (hence it's a template argument).

For instance, on my compiler (GCC 4.8.4), for this example program:

  #include <iostream>
  #include <vector>
  #include <array>

  int main()
  {
        std::vector<int> v_ten(0, 10);
        std::vector<int> v_twenty(0, 20);
        std::array<int, 10> a_ten{0};
        std::array<int, 20> a_twenty{0};

        std::cout << "sizeof(v_ten): " << sizeof(v_ten) << std::endl;
        std::cout << "sizeof(v_twenty): " << sizeof(v_twenty) << std::endl;
        std::cout << "sizeof(a_ten): " << sizeof(a_ten) << std::endl;
        std::cout << "sizeof(a_twenty): " << sizeof(a_twenty) << std::endl;

        return 0;
  }
produces this output:

  sizeof(v_ten): 24
  sizeof(v_twenty): 24
  sizeof(a_ten): 40
  sizeof(a_twenty): 80
edit: added comment about alignment.
Re:Java’s new garbage collector promises low pause times on ... by dbaupp on Thursday August 30, 2018 @06:52PM (#17881513)
The parent is correct. std::vector is a value type in the sense that everyone else is talking about... All types in C++ are, as their members are not inherently stored on the heap. C++ (and C, and Rust) has types that behave like what other languages call value types, even if there is not any other sort of type.

The mention of array in the parent was referring to the dynamic storage of a vector, not std::array.

Re:Java’s new garbage collector promises low pause times on ... by stcredzero on Thursday August 30, 2018 @06:22PM (#17881322)
When Java was created, they wanted to get rid of that complexity, so primitives always go on the stack and objects always go on the heap.*

This was inherited from Smalltalk. Everything was an "object reference," with tag bits and all, but SmallInteger was optimized by using the low 16 bits of the object reference pointer to store the SmallInteger. So you can think of Smalltalk as having SmallInteger as a "primitive" type with pass by value.