News for hackers. Stuff that matters.
Re:GC Tuning Confessions of a Performance Engineer by banachtarski on Friday May 29, 2015 @07:29PM (#9628160)
How many engineers work on, say, the Unreal Engine. The linux kernel? Windows? Do you think any of that stuff would work even marginally well running on the JVM (assuming the JVM had native driver support)?

I don't quite follow your claims, nor do I put any stock in the tenure of your programming career.

As for the off-shoring point, off-shoring is going to create questionable quality code regardless of the programming language.

Re:GC Tuning Confessions of a Performance Engineer by pjmlp on Saturday May 30, 2015 @01:24AM (#9629128)
> How many engineers work on, say, the Unreal Engine. The linux kernel? Windows?

So you just provided me examples where developers tend to be highly skilled just to get a foot in the door.

Yet, the CVE list gets updated regularly with memory corruption exploits for them.

https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=double+free

Or if you prefer, just for Linux

https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=linux+doubl...

> Do you think any of that stuff would work even marginally well running on the JVM (assuming the JVM had native driver support)?

Actually, are you aware that some military weapon systems are being driven with JVMs?

Anyway automatic memory management doesn't mean automatically a JVM, there are quite a few other ways.

Another fun fact, Unreal Engine and Windows are written mostly in C++. A language considered too bloated and slow to be usable for anything serious by mainstream developers in the early 90's.

Also both use automatic memory management on their systems. Unreal has a kind of GC library. Windows nowadays has COM almost everywhere with reference counting.

> I don't quite follow your claims, nor do I put any stock in the tenure of your programming career.

Apparently some well known Fortune 100 companies and research institutes had another opinion.

> As for the off-shoring point, off-shoring is going to create questionable quality code regardless of the programming language.

The point being that those projects tend to go for cheap developers, so no, not everyone can deal with manual memory management.

Re:GC Tuning Confessions of a Performance Engineer by banachtarski on Saturday May 30, 2015 @04:42AM (#9629500)
> So you just provided me examples where developers tend to be highly skilled just to get a foot in the door.

Yep these are people I want to work with :)

> Yet, the CVE list gets updated regularly with memory corruption exploits for them

Security is not the concern of every piece of software. You can harden the pieces you care about. Honestly, not all of us are doing crypto, and for that, the recommendation is to leverage an existing library and sandbox it anyways (regardless of memory management strategy).

> Actually, are you aware that some military weapon systems are being driven with JVMs?

God help them

> Another fun fact, Unreal Engine and Windows are written mostly in C++. A language considered too bloated and slow to be usable for anything serious by mainstream developers in the early 90's.

I was developing in C and C++ in the 90s. C++ is a strict subset of C. The criticisms made no sense then and they make no sense now. C++03 was arguable an small incremental improvement over C++98 but C++11 is a huge step function in usability without sacrificing performance (which Rust has taken many good cues from). The point is, if something bad is happening, I can look at the dissassembly and see exactly what's going on. I can't do this with Java, Python, Ruby, etc.

> Also both use automatic memory management on their systems. Unreal has a kind of GC library. Windows nowadays has COM almost everywhere with reference counting.

The scripting engine in Unreal supports GC if you want it. It's trivial to embed Lua or something. That's part of the point. For the non-performance sensitive bits, sure do whatever. The COM interface is terrible, and a necessary evil for those in the industry. There's a reason DirectX12 is looked forward to. I don't want Windows reference counting my things any more unless I tell it to.

> Anyway automatic memory management doesn't mean automatically a JVM, there are quite a few other ways.

... yes, I honestly can't imagine anyone in this thread would think otherwise. Heck, I've written a GC for a homebrew scripting language.

> Apparently some well known Fortune 100 companies and research institutes had another opinion.

My point is that how long you worked is sort of irrelevant in an actual academic discussion. Where you worked is just as irrelevant.

Re:GC Tuning Confessions of a Performance Engineer by pjmlp on Saturday May 30, 2015 @05:44AM (#9629605)
> Yep these are people I want to work with :)

Which comes back to my point that only developers able to get hired at that level are able to do manual memory management in large scale.

And even then, they resort to automatic memory management at higher levels in their stacks.

> I can look at the dissassembly and see exactly what's going on. I can't do this with Java, Python, Ruby, etc.

Just get a commercial JVM compiler, JIT Watch, Intel Amplifier. All of them provide the option to show the generated machine code.

> There's a reason DirectX12 is looked forward to. I don't want Windows reference counting my things any more unless I tell it to.

Better stop developing for Windows then.

The Universal Windows Application model is the continuation of WinRT, an evolution of COM.

In case you missed the note, DirectX 12 is still based on COM.

Also the new User Driver Model is based on COM.

Re:GC Tuning Confessions of a Performance Engineer by banachtarski on Saturday May 30, 2015 @01:17PM (#9630850)
Yea still COM based but part of what is being removed is a huge swath of underlying automatic reference counting for a huge number of things. Constant buffers resources etc.

Sure you can look at machine code generated from a virtualized language but I'd have a tough time understanding it in the context of the whole runtime, let alone being able to practically do anything about it.

Re:GC Tuning Confessions of a Performance Engineer by pjmlp on Saturday May 30, 2015 @02:10PM (#9631116)
> Sure you can look at machine code generated from a virtualized language but I'd have a tough time understanding it in the context of the whole runtime, let alone being able to practically do anything about it.

It is no different than using -S and check what happens for several code patterns.

Re:GC Tuning Confessions of a Performance Engineer by banachtarski on Saturday May 30, 2015 @02:54PM (#9631323)
Look man, I don't know what to tell you except that you view code fundamentally differently. My point is that yes, obviously everything gets turned into x64/x86 assembly at some point, and that yes, I'm sure seeing it is very easy. But if I need to prevent a GC pause from happening at some point in the frame my hands are tied. Debating this is like pulling teeth, and I'm inclined to say that if you can't follow at this point, just use your GC everywhere.
Re:GC Tuning Confessions of a Performance Engineer by pjmlp on Saturday May 30, 2015 @03:45PM (#9631515)
This whole thread started by my statement that I haven't seen large projects without errors doing manual memory management.

I doubt very seriously that anyone can write code in large teams where Valgrind will state there aren't double frees, bad frees, or dangling pointers happening.

Or a run with Coverity will state everything is nice and shinning.

Once upon a time I had to write a tool in a well known particle accelerator research institute to track down C++ memory errors in a multi-thread environment for cluster algorithms used in data analysis. As one common problem on that specific team was plugins bringing the cluster down due to memory corruption.

So I also do know one or two things about manual memory management.

And with what I know, I rather use automatic memory management, be it in the form of GC, RC, affine types or dependent types.

Re:GC Tuning Confessions of a Performance Engineer by banachtarski on Saturday May 30, 2015 @06:27PM (#9632007)
Dangling resources are common in any language. Ports don't get reclaimed, file descriptors are opened and never closed... the same principles apply.

Its funny since I actually started writing code for particle accelerator research as well, although it was a mixture of terrible C and F77. Physicists and mathematicians are good at their respective fields, but not necessarily coding sadly (I used to be both). Would the code have been better with a different language? Doubtful. They used literally zero established patterns and wrote everything with copy paste and with zero regard for what the machine actually did.

To each their own though. I'm glad your projects and line of work afford you the ability to work with a GC. Mine do not.