News for hackers. Stuff that matters.
Re:Using D to Create the World’s Fastest File System by nonsince on Wednesday December 05, 2018 @12:02PM (#18610119)
The main reason the way I see it is that a lot of people who still use C and C++ need maximum speed, and D is only memory-safe with its garbage collector enabled. This affects latency and adds overhead. For many cases using a garbage collector is fine, but D was trying to target video games and other spaces where the GC was not acceptable. So D allows you to turn the GC off, but then it's just a vastly less-mature C++ with similar safety footguns and worse library support. Rust actually fills a space that wasn't filled by anything else - memory safety without a garbage collector. You can write pretty ergonomic code in Rust (similar to how code would look in a dynamic and/or GC'd language) just like D, but where in D you would have to choose between highest-possible efficiency and safety, Rust gives you both at the same time.

This isn't to say that Rust is better than D, but it is to say that Rust targets a gap in the market whereas D is trying to have the same tradeoffs as C++ but be better, which is nowhere near as compelling for most developers who've invested a lot of time into learning C++. C++ developers don't use D for the same reason that most people don't switch from qwerty to dvorak even though it's technically more efficient. It's not such a qualitative change that it's worth the retraining.

Re:Using D to Create the World’s Fastest File System by WalterBright on Wednesday December 05, 2018 @03:02PM (#18611879)
> D is only memory-safe with its garbage collector enabled.

This is technically correct, but not pragmatically correct. D doesn't have every memory unsafe operation plugged, but it's pretty darned close, and does have the major ones covered (like array bounds checking, alternatives to pointers, RAII, etc.).

> library support

D has excellent library support.

Re:Using D to Create the World’s Fastest File System by kllrnohj on Wednesday December 05, 2018 @03:41PM (#18612308)
> (like array bounds checking, alternatives to pointers, RAII, etc.).

Yeah but C++ has all of that, too, as well as a vastly broader ecosystem in the space of non-GC'd, ultra-high-performance languages.

Re:Using D to Create the World’s Fastest File System by WalterBright on Wednesday December 05, 2018 @03:45PM (#18612348)
C++'s array bounds checking is there only if you use vector<>, not regular arrays.
Re:Using D to Create the World’s Fastest File System by kllrnohj on Wednesday December 05, 2018 @04:00PM (#18612482)
And D's bounds checking is only there if you don't use pointers or @trusted.

fyi std::array also has bounds checking, you're not limited to vector<> to have that.

Re:Using D to Create the World’s Fastest File System by WalterBright on Wednesday December 05, 2018 @04:22PM (#18612704)
D will give you compilation errors in @safe mode if you try to index off of a pointer.
Re:Using D to Create the World’s Fastest File System by jordigh on Wednesday December 05, 2018 @02:26PM (#18611590)
Is the D GC actually that bad? Do we have benchmarks or something to prove that the GC means D is dead in the water? I always hear about this hypothetical danger of GC but my D usage hasn't found it and my D programs typically match or outperform my C++ programs. I mostly use it for numerical code, so maybe that's why I'm not feeling the pain of the GC?

To address your final point, I find D vastly superior to C++. Immensely, tremendously, bigly superior to C++. It's so nice, it's so pleasant, it's so easy. It's not an incremental change, it's a whole new world of no segfaults, no template metaprogramming, beautiful error messages, beautiful compile-time calculations.

Re:Using D to Create the World’s Fastest File System by kllrnohj on Wednesday December 05, 2018 @03:47PM (#18612367)
There is no such thing as a "good GC" in this context. Game engines go to great lengths to do things:

1) Have bounded memory usage 2) Have consistent frame times

These goals are critical to maintain a consistent framerate on fixed-memory devices (aka, consoles). GCs, by their very nature, are not compatible with either of those goals. They need lots of spare memory to go fast, and they cause hiccups when collection happens. They are a non-predictable, non-consistent load.

If you're not working on a problem that is a sustained, consistent workload that needs sustained, consistent result generation then no, you probably won't see the same GC problems. GCs work great when the work is bursty or when the work has no latency associated with it. There's an awful lot of programs that fit in that model. Games just aren't one of them.

Re:Using D to Create the World’s Fastest File System by DblPlusUngood on Wednesday December 05, 2018 @04:40PM (#18612870)
How are GCs not compatible with bounded memory use? Though many GCs size allocation arenas proportionally to the live data, there is no fundamental reason why the allocation arenas couldn't be fixed-size (of course, they must be large enough for good performance, etc.).
Re:Using D to Create the World’s Fastest File System by p0nce on Wednesday December 05, 2018 @04:51PM (#18612980)
And yet there are GCs in Unity and Unreal Engine, the most popular game engines of today.
Re:Using D to Create the World’s Fastest File System by monocasa on Wednesday December 05, 2018 @05:07PM (#18613130)
Only the scripting code is GCed. The core engine in both cases is C++ with manual memory management for the reasons stated above.
Re:Using D to Create the World’s Fastest File System by pjmlp on Thursday December 06, 2018 @04:38AM (#18616587)
On Unity's case with some subsystems in the process of being replaced by HPC#.

And in Unreal you can use GC in whatever components you feel like, it is a matter of weighting where it makes sense.

Re:Using D to Create the World’s Fastest File System by kllrnohj on Thursday December 06, 2018 @09:20AM (#18618210)
> On Unity's case with some subsystems in the process of being replaced by HPC#.

They are converting their C# code to HPC# which specifically does not do GC allocations.

https://youtu.be/NF6kcNS6U80?t=886

  HPC#:
   * no class types
   * no boxing
   * no GC allocation
   * not exceptions for control flow
They are building a compiler to eliminate the GC from the GC'd language they were using.
Re:Using D to Create the World’s Fastest File System by pjmlp on Thursday December 06, 2018 @09:51AM (#18618433)
Once upon a time, C code for games engines looked like this.

    void do_stuff(void) {
        asm {
        }
    }
Hardly any C code in sight, and most of the stuff could equally have been coded in MASM/TASM with their higher level macros.

Eventually game devs migrated to proper C.

A couple of years later, C++ code in game engines, was not even "C with Classes", rather C compiled with C++ compiler.

Eventually game devs migrated to proper C++.

Looking at the past, which I was partially part of, I see HPC# as a stepping stone into regular C#, in a similar vein that C code of yore was full of inline Assembly.

It is of course a matter of opinion, however I like to look at the past to see where the future leads to.

So lets see who's is right in about 10 years, regarding how game engines get implemented.

Re:Using D to Create the World’s Fastest File System by kllrnohj on Thursday December 06, 2018 @01:54PM (#18620912)
Except they are stepping AWAY from C# here, not towards it.

And the game-industry as a whole is moving to data-oriented design, which tends to be hard to do at best (if not downright impossible) in most GC'd languages.

Regardless the transition from C to C++ has no performance cost (often a performance advatnage), so I'm not sure why you're bringing that up as some significant migration in this case? It doesn't support any argument for a transition from C++ to C# or similar, as that comes with a huge performance cost. Particularly in an era where CPU performance hasn't improved for almost a decade now.

Re:Using D to Create the World’s Fastest File System by pjmlp on Thursday December 06, 2018 @03:46PM (#18622044)
Modern CPU performance gets explored by taking advantage of multi-core, and GC is a very helpful tool for writing lock-free data structures.

Additionally dataflow languages are better suited for GPGPU programming than plain old C and C++, as Google is exploring with XLA on Tensorflow, and game engines with graphical tooling for shader generation.

Again, lets come back in 10 years and see who was actually right.

Re:Using D to Create the World’s Fastest File System by swsieber on Wednesday December 05, 2018 @04:59PM (#18613060)
Electron is a very popular application framework today.
Re:Using D to Create the World’s Fastest File System by pjmlp on Wednesday December 05, 2018 @12:35PM (#18610449)
On what concerns video games, all major engines do use GC on gameplay code, including variations of C++ GCs.
Re:Using D to Create the World’s Fastest File System by jdright on Wednesday December 05, 2018 @01:23PM (#18610954)
emphasis on gameplay code. And even there, if you target 60fps (sure a minority of developers), then GC is a liability at best.
Re:Using D to Create the World’s Fastest File System by pjmlp on Wednesday December 05, 2018 @01:53PM (#18611243)
But that is exactly the point. One should not constrain productivity just based on hypothetical goals, web scale and such similar arguments.

Doing the next Fortnite, Crysis or ground breaking AR/VR/Ray Tracing? Sure, every ms/byte counts.

Doing a typical Flash like casual game, game prototypes at Ludum Dare, participating at IGF? Having a GC around isn't the biggest concern.

Re:Using D to Create the World’s Fastest File System by derefr on Wednesday December 05, 2018 @06:28PM (#18613830)
> Doing a typical Flash like casual game, game prototypes at Ludum Dare, participating at IGF? Having a GC around isn't the biggest concern.

But for those cases, what would move you to use D instead of an entirely memory-safe language like Python or Lua?

To justify using D for games, you'd need to find a use-case where speed (e.g. targeting 60fps) is high-priority enough to be writing a lot of low-level code; and yet, where "having a CG around" still won't cause problems. Can you think of one?

Re:Using D to Create the World’s Fastest File System by pjmlp on Thursday December 06, 2018 @02:42AM (#18616161)
Yes, because it compiles to native code and has relatively fast compile times.

Some AAA studios do use D instead of a C++ based script language exactly for that.

Re:Using D to Create the World’s Fastest File System by jdright on Monday December 10, 2018 @01:43PM (#18650149)
Only one studio, no more. And I hear there are some questions about this decision...
Re:Using D to Create the World’s Fastest File System by faissaloo on Wednesday December 05, 2018 @01:27PM (#18610990)
Actually GC can decrease overhead depending on your use of memory since it only frees when memory when it's needed.
Re:Using D to Create the World’s Fastest File System by skohan on Thursday December 06, 2018 @07:13AM (#18617323)
> GC ... only frees when memory when it's needed.

This is actually one of the worst features of GC where game development is concerned. Memory usage isn't the main problem to be solved in game engines, it's consistency of execution time.

In that context, even if you're spending more resources on memory allocation/deallocation overall, it's much better to know that you can consistently fit that overhead into the 16 milliseconds you have for this frame rather than having it mediated by a system you do not have direct control over.

There are also other advantages to having more direct control over memory management than is typically allowed by garbage-collected languages. For instance, CPU cache-coherency can be a major performance concern when dealing with the type of computation required for games. Without being able to lay out memory and deal with memory in a precise way, there are whole categories of optimization which are not really possible.

Re:Using D to Create the World’s Fastest File System by monocasa on Wednesday December 05, 2018 @01:49PM (#18611203)
"Overhead" is a term with many definitions.

Non-conservative GCs use a lot of extra metadata to be able to figure out what's a pointer and what isn't. Conserative GCs leak.

In both cases GCs cut into worse case latency.

Re:Using D to Create the World’s Fastest File System by groestl on Thursday December 06, 2018 @01:02AM (#18615792)
> Conserative GCs leak.

Yes, but usually the leak is bounded [0], although I'm not aware they evaluated real-world gaming engines. But I guess frame jitter is the bigger problem there.

[0] https://www.hboehm.info/gc/bounds.html