I believe what he's referring to is also called AoS(Arrays of Structures) vs SoA(Structure of Arrays).

If you're iterating over one or two values in a struct via AoS it's very painful from a memory caching standpoint since you're only getting sizeof(member) / sizeof(struct) efficiency.

In a SoA situation all your data for members is tightly packed so you usually get better cache coherency.

That said the best approach is actually looking at your data access patterns and packing your data appropriately. Unfortunately some languages don't have value types(most managed languages except C#). This is why C/C++ is usually faster than managed languages, they can get close on generated code but can be off by a factor of 50x in memory access performance.

> This is why C/C++ is usually faster than managed languages, they can get close on generated code but can be off by a factor of 50x in memory access performance.

This is a common misconception that is a result of Java's current state. Control over memory layout and garbage collection are two completely orthogonal issues. At the moment, Java just happens to be both GCed and to afford little control over memory layout. Currently, the HotSpot team (HotSpot is the name of OpenJDK's JVM) is working on project Valhalla and project Panama, two efforts scheduled for Java 10 and intended to give the JVM all the memory layout control you need.

Not completely.

Most GCs want to be able to compact the heap, that means they lean in favor of reference based models by default.

It's not exclusive like you mention but to my knowledge C# is the only language with explicit guarantees. I was aware of the effort around Java's value types, however how long till we can reasonably see this in production?

You can do this in Java today but it involves lots of nasty ByteBuffer manipulation(in which FlatBuffers is fantastic for). You'll still pay for the conversions from bytes to the actual types you want.

If you're dealing with these types of performance problems it's best to treat them with a language that's well suited to deal with them. There's nothing wrong with using Java for higher level business logic and delegating the heavy lifting to a stack that's designed to deal with them.

> Most GCs want to be able to compact the heap, that means they lean in favor of reference based models by default.

I don't see how this follows. Copying GCs do compact the heap, but when you use values you're basically saying "these two things go together". You're only making life easier for the GC (well, you're also creating larger objects, but that might just require a small adjustment of the GC strategy).

> how long till we can reasonably see this in production?

Four years probably... Still, that doesn't change the fact that layout and GCs are orthogonal.

Expanding on my point a bit, I'm not arguing that in theory GC and memory layout are orthogonal.

Just that in practice you don't see memory layout in languages that are managed. Since I'm usually not in a habit of building a new languages to solve problems it's something worth understanding when making a technical selection on a language.

D, Modula-3 and the Oberon family are two examples of value types by default with a GC enabled heap.

Eiffel also offers value types (aka expanded), but it is reference by default.

> This is why C/C++ is usually faster than managed languages, they can get close on generated code but can be off by a factor of 50x in memory access performance.

This only happens because the Pascal branch of languages with GC, sadly failed in the mainstream (for several reasons).

Modula-3, Eiffel, Oberon, Oberon-2, Component Pascal all offer the same data access patterns and packing available to C and C++.

Many programming languages put boxes around the bits that the CPU actually operate on. These boxes often store things like reference count and type (or a dispatch table for object-oriented languages with single-dispatch). That thing inside a box that is the actual value is called an atom.

Those boxes have a lot of overhead (in PHP it's around 140 bytes!), and because a major identified problem is the waste of valuable cache space and CPU time book-keeping these boxes, a lot of JIT and compiler research is about how to eliminate the boxes.

You can see why they do this when considering a program like:

    $a = array();
    for($i = 0; $i < 100000; ++$i) $a[$i] = rand()*65536;
    sort($a);

which might take 200k (and fits easily in cache) in C/C++ but 15 megabytes (and doesn't) in PHP.

However.

Eliminating those boxes has proven to be very difficult, and while I have heard great things about Java 10, I am reserved because I also heard great things about Java 5. And Self. I do not expect JIT/compilers will get smart enough to actually solve this problem in my lifetime.

On the other hand, array programming languages allow you to get that space-overhead that C/C++ has in an interactive (and usually interpreted) language, so it so happens that you don't have to choose between a nice interpreted programming language with introspection, automatic memory management, and so on, and high performance, but it does mean you have to program the arrays instead of the atoms.

I do not understand how that source file is "OO business logic".