News for hackers. Stuff that matters.
Re:The reference D compiler is now open source by softinio on Friday April 07, 2017 @12:53PM (#14061347)
Whats special about D? why should i learn it?
Re:The reference D compiler is now open source by grok2 on Friday April 07, 2017 @04:26PM (#14063227)
I think one of the very good features of D is that it provides a lot of infrastructure that helps with lots of little generic things -- things that are not really needed in the language, but help reduce the amount of code you write or simplify common actions you take. It seems like Walter (and all the other contributors) have distilled their experience writing programs into creating a language that helps a lot with some of these things.

Stuff that I particularly like.

Assert/Enforce: http://ddili.org/ders/d.en/assert.html , Unit testing: http://ddili.org/ders/d.en/unit_testing.html , Contract programming help: http://ddili.org/ders/d.en/contracts.html, and http://ddili.org/ders/d.en/invariant.html , Scope (love this): http://ddili.org/ders/d.en/scope.html

And the other usuals -- templates, mixins, etc....

BTW, I mostly use D as a better C than as a better C++...

Re:The reference D compiler is now open source by vram22 on Sunday April 09, 2017 @08:55AM (#14072045)
Good list.

Another good feature:

Interfacing to C is somewhat easy at least for the basics. (I haven't looked at advanced cases yet, but maybe someone else can comment on that.)

https://dlang.org/spec/interfaceToC.html

This is a great feature IMO, because it allows you to (re)use the huge number of existing C libraries out there.

Here is a simple example:

Calling a simple C function from D - strcmp:

https://jugad2.blogspot.in/2016/09/calling-simple-c-function...

There's a handful more small examples of a few things D can easily do, here on my blog:

https://jugad2.blogspot.in/search/label/DLang

(And of course, there are many more at the DLang tour site - https://tour.dlang.org/ )

Also, there are a few good videos about D (alone) and being discussed along with other languages, at my blog's DLang posts link above.

I'll just put some post titles here to give a quick idea:

Simple parallel processing in D with std.parallelism

Using std.datetime.StopWatch to time sections of D code

Read from CSV with D, write to PDF with Python

Command line D utility - find files matching a pattern under a directory

min_fgrep: minimal fgrep command in D

num_cores: find number of cores in your PC's processor

Func-y D + Python pipeline to generate PDF

file_sizes utility in D: print sizes of all files under a directory tree

deltildefiles: D language utility to recursively delete vim backup files

[DLang]: A simple file download utility in D

Getting CPU info with D (the D language)

Re:The reference D compiler is now open source by vram22 on Sunday April 09, 2017 @06:29PM (#14074679)
Just added another D post to that list:

Porting the text pager from Python to D (DLang)

https://jugad2.blogspot.in/2017/04/porting-text-pager-from-p...

Re:The reference D compiler is now open source by gmfawcett on Friday April 07, 2017 @12:58PM (#14061390)
Except for perhaps Lisp languages, almost no language makes compile-time computing and code-generation so easy. This allows for some really powerful language features that can be designed as libraries, and puts this power in the hands of "regular developers" rather than only in the hands of template wizards.
Re:The reference D compiler is now open source by jordigh on Friday April 07, 2017 @01:15PM (#14061545)
Yeah, I really love that at compile time you can meta-program in almost the same language that you use for normal run-time things. I think the comparison to lisp is apt. It almost feels like lisp macros.

Think C++ template metaprogramming but much, much easier and thus, seemingly more powerful. It's not that you couldn't do the same in C++, but D's metaprogramming is so much more accessible that it makes you want to use it.

Re:The reference D compiler is now open source by jacques_chirac on Friday April 07, 2017 @01:18PM (#14061569)
There's plenty you can do with D but not C++, see regex, bitfields, swap member-by-member with checks for mutual pointers, the pegged library for grammars, etc etc.
Re:The reference D compiler is now open source by jordigh on Friday April 07, 2017 @01:34PM (#14061726)
I don't know, Boost has me convinced that even things like regexes in C++ templates could be possible, if your compiler has a big enough stack to handle very deep template recursions. It's theoretically possible to build a regex parser in C++ templates, right? It would be horrible, but possible.
Re:The reference D compiler is now open source by qznc on Friday April 07, 2017 @01:57PM (#14061978)
With D it is possible and not horrible and already done. ;)

Talk: http://dconf.org/2014/talks/olshansky.html

Re:The reference D compiler is now open source by humanrebar on Friday April 07, 2017 @02:39PM (#14062377)
> It's theoretically possible to build a regex parser in C++ templates, right?

You'd probably do large portions of it in "constexpr" types and functions rather than relying on recursive metaprogramming techniques.

They are also planning on adding overloading based on whether a function is constexpr, which is important if you want the same library to support both compile-time and run-time matchers.

So it's all theoretically possible already. But D has the advantage here in that it's not just possible but usable.

Re:The reference D compiler is now open source by dbcurtis on Friday April 07, 2017 @01:26PM (#14061651)
For someone who knows zero about D, but is a total Pythonista, how would you compare the meta-programming facilities?
Re:The reference D compiler is now open source by jordigh on Friday April 07, 2017 @01:31PM (#14061692)
I guess the best comparison to Python would be the dunder methods. You know how in Python a class really is just a dict, right? Like, foo.bar is pretty much syntactic sugar for foo.__get_attribute__(foo.__dict__['bar']) in most cases. Or how a + b is sugar for a.__plus__(b).

Well, imagine that all of the things you can do in Python by messing with dunder methods, function decorators, or metaclasses could be precomputed by a compiler and would not even execute at all during runtime. It makes everything really fast at runtime and easy to precompute at compile time.

Re:The reference D compiler is now open source by gmfawcett on Friday April 07, 2017 @01:48PM (#14061879)
Yes, and plus you get type safety with all of that! When I'm writing in D, my editor marks all the type errors (and typos) in my code, almost as fast as I make them. The DMD compiler is so quick that there's almost no lag. This leads not only to a rapid development cycle, but a much higher level of reassurance (than I've had when writing, e.g., metaclass hacks in Python).
Re:The reference D compiler is now open source by dbcurtis on Friday April 07, 2017 @02:20PM (#14062214)
Well, overriding dunder methods isn't really metaprogramming. It's just method inheritance/override.

Is there anything in D similar to decorators? How does the registration pattern work in D, for instance?

Re:The reference D compiler is now open source by gmfawcett on Friday April 07, 2017 @02:34PM (#14062327)
Do you mean like this kind of registration decorator?

https://github.com/rejectedsoftware/vibe.d/blob/master/examp...

Decorators (in D, 'user defined attributes', or UDA) work differently. It's not a function-composition feature, but more of a tagging feature. You write a class, function, etc., tag it with custom attributes; and then have a separate compile-time function walk over your code, find the decorators, and augment the code based the meaning of the tags. (I used to wish that D had adopted Python-style decorators, since they are easy to reason about and implement, but I can see the logic of the more general UDA system that they adopted.)

In practice, though, you would often use templates to achieve the same effect. Given a memoize template (really, just a memoize function), and an expensive-computation function,

    auto fastComputer = memoize!expensiveComputer;
produces roughly the equivalent of

    @memoize
    def expensiveComputer(): ...
but with opportunities for compile-time optimization.
Re:The reference D compiler is now open source by dom96 on Friday April 07, 2017 @01:47PM (#14061874)
Anybody out there that has experience with both Nim and D? I am curious how they both compare in terms of metaprogramming.
Re:The reference D compiler is now open source by gmfawcett on Friday April 07, 2017 @02:04PM (#14062053)
I've dabbled with Nim. It's also a great language with great metaprogramming features. Plus a fast compiler that generates very lean code. (The real reason I stick with D over Nim, above all else, is RAII -- once you have it, it's really hard to live without. Okay, that, and also ranges, and array/string slices... they are fantastic to work with. Nim's got a better GC story right now, IMO, but there's a lot of activity in the D community to become more competitive in that space.)

I found that some of the Nim's MP features, in particular AST macros, are a little harder to work with than D's templates and compile-time function evaluation. You get a lot of flexibility, but the cost is high. I'm not a big fan of how "mixin" is used in D to splice source-text into a generated function, it's certainly less principled than an AST transformation, but in practice the resulting code tends to be concise and easily read, whereas the AST-macro approach introduces a lot of accidental "noise" and complexity. The complexity raises the bar for reaching for a compile-time solution, where in D it seems equally as natural to write compile-time code as it does to write regular code. (Not to pick on a strawman, though -- AST macros are not Nim's only MP tool.)

The Nim MP feature I never really played with was the rewrite rules. They seem interesting in theory, but maybe a little too magical for my liking!

I have a lot of respect for Nim. I am glad we live in a world with so many options. Like Walter said in an earlier comment, it's an embarrassment of riches. :)

Re:The reference D compiler is now open source by theprotocol on Friday April 07, 2017 @03:17PM (#14062713)
Thanks for the writeup. I've just been evaluating Rust, D, and Nim, and reading about your experience has been helpful.

Can you elaborate on how D is planning to sort out the garbage collection? Nim's GC is extremely fast and thread local, and can be disabled without breaking libraries (according to the author, it does something with memory regions that I haven't 100% grasped yet).

I've googled about D's garbage collector and it's apparently been discussed as the language's biggest flaw since 2013, but I can't find any information whatsoever on what's being done in that regard.

Re:The reference D compiler is now open source by gmfawcett on Friday April 07, 2017 @03:51PM (#14063000)
Like Nim, you can also disable GC in D, and you can avoid GC altogether by not allocating GC'd memory (GC is only ever triggered at allocation time), or at least by preallocating and then disabling the collector. Raw allocation is always possible, and you can "emplace" D structs and classes into unmanaged memory.

The collector itself isn't being improved as far as I know -- at least I haven't seen any initiatives mentioned recently with that goal. In fairness, I haven't been following the community activity very closely in the past few months, but I think that's accurate. Nim definitely has a technical advantage re: its GC implementation.

The bigger movement has been the "@nogc initiative", which started with adding a @nogc attribute to the language (the compiler can verify that a function tagged with @nogc, and all of its callees, do not allocate GC memory). There is an ongoing initiative to make more of the Phobos library @nogc-compliant, to take advantage of this feature. There has also been a lot of work on custom memory-allocators [1], and I think the plan is to incorporate into Phobos where it makes sense, so you can have functions which take custom allocators, have thread-local allocators, etc.

https://dlang.org/phobos/std_experimental_allocator.html

I don't speak for the community or the dev team, but I think the long-term goal is to make the GC a feature that is available when you want it, but that isn't a dependency for using the standard library. Either through custom allocators or through @nogc guarantees, you'll be able to ensure that your program's memory management is deterministic.

Re:The reference D compiler is now open source by theprotocol on Friday April 07, 2017 @04:10PM (#14063124)
Thanks for the info. It's pretty hard to get clear, updated info on these languages as they have yet to gain that much traction (and they tend not to be backed by big entities that have a PR budget).

For anyone else who's evaluating D and looking into its GC situation, the most recent blog post on Dlang.org (https://dlang.org/blog/2017/03/20/dont-fear-the-reaper/) seems to embrace the presence of the GC, but also ends by saying the next blog post will describe how to go without the GC. So there is indeed awareness/activity on that front!

Re:The reference D compiler is now open source by he_the_great on Monday April 10, 2017 @05:40PM (#14082803)
There has been work done to improve the collector but nothing has made it into mainline. For example one implementation was missing parts for Windows but worked in Linux thus it couldn't be pulled in and used.
Re:The reference D compiler is now open source by bachmeier on Friday April 07, 2017 @01:34PM (#14061719)
If you want to work with C code, it is an excellent choice. I use it for numerical computing. It's an easy language to learn, no need to worry about memory management if you don't want/need to, generally good syntax, nice compile time features. Overall the best "better C" in my opinion.
Re:The reference D compiler is now open source by kaleidic on Friday April 07, 2017 @05:58PM (#14063812)
See bachmeier work on D to R integration.
Re:The reference D compiler is now open source by amelius on Friday April 07, 2017 @01:31PM (#14061691)
Also, how does D compare to Rust?

(If I'm going to learn a new system programming language, which one should I pick?)

Re:The reference D compiler is now open source by theprotocol on Friday April 07, 2017 @03:30PM (#14062820)
My early impression with Rust is that it's more ambitious than it is capable of delivering. Lifetime annotations in particular are incredibly ugly. It's the kind of thing that makes me think: "...ok, so this is why other languages don't simply do static lifetime checks" and I guess I expected Rust to be the solution for that problem; it is instead not a solution, but the deliberate decision: "let's do static borrow checking" and that was at odds with what I expected from it.

In short, I thought Rust was "we solved the pain of this difficult thing." However, it is not really a novel solution to the difficult thing, but simply a decision to undertake it.

I've now moved to evaluating Nim and D, and while I've barely dabbled in them (mostly going through the docs and writing some simple example-like code), I can at least say that Rust is likely to be much more controversial and polarizing than these languages. I recommend giving it a try - go through the official "book" which serves as the main documentation. If you survive lifetime annotations without finding them too obnoxious, then you'll probably like Rust.

Personally, I find Rust's extreme rigorousness (and the laboriousness that follows from that) makes it very niche, and I find myself rather unexcited about using it for anything.

Re:The reference D compiler is now open source by burntsushi on Friday April 07, 2017 @05:26PM (#14063631)
Note that the vast majority of lifetimes in Rust are elided so that no explicit annotation is required. When the compiler can't figure out the annotations, or if you're declaring a type that contains a borrow, then you need annotations.

The annotations have, for the most part, faded into the background for me. I don't find them to be particularly pervasive in the code I write, and when I do need them, it's usually to fix a reasonably straight-forward case where the compiler failed to elide them. With that said, in the years I've been using Rust, I have committed one or two lifetime-related bugs. (No memory unsafety resulted, of course, but it did result in data being annotated with a shorter-than-actual lifetime.)

> In short, I thought Rust was "we solved the pain of this difficult thing."

Part of the pain that Rust purports to solve is the significant reduction (and debugging thereof) of memory unsafety errors. In return, you must deal with the pain of an ownership and borrowing system, which might present challenges to patterns you may have used in another language. The benefit is that you have a compiler to tell you when you've mis-stepped instead of an end user filing a bug report (or worse).

Re:The reference D compiler is now open source by theprotocol on Saturday April 08, 2017 @08:28AM (#14066591)
I agree with your characterization of the benefits and tradeoffs of Rust. I think that regardless of whether it solves the pain point, at the very least there needs to exist something that undertakes static lifetime checking. I respect Rust from a distance. It's not in line with my particular needs and/or expectations, but I'm glad it exists, because something needs to exist which does what Rust does, and I'm not aware of anything else that does it.

Other than lifetime annotations, I have a couple other niggling issues with it but I suspect it's because I haven't fully grasped certain things yet, and so I'll refrain from commenting on them for the moment.

Re:The reference D compiler is now open source by Rusky on Friday April 07, 2017 @02:16PM (#14062175)
D has been around longer, has powerful compile-time metaprogramming tools, and has a very fast compiler. However, it has a garbage collector, and is not entirely memory-safe by default (though is beginning to optionally incorporate some ideas from Rust [edit: or not], maybe making that easier once you put in the work). It seems to follow C or C++ in what sorts of things it makes language-level features.

Rust is newer (though being used in some big projects like Firefox/Servo, Visual Studio Code's search functionality now uses it by default, etc), has (afaiu) a more powerful type system, and is entirely memory-safe by default. However, this makes it somewhat harder to learn at first, and it also has a relatively slow compiler. It follows ML-style languages a bit more, with things like pattern matching and sum types built into the language.

Personally, I prefer Rust, because it feels like a stronger foundation with the ML-like type system and no GC. YMMV though, depending on what is important to you.

Re:The reference D compiler is now open source by WalterBright on Friday April 07, 2017 @02:23PM (#14062241)
D's memory safe notions are very different from Rust's.
Re:The reference D compiler is now open source by dbaupp on Friday April 07, 2017 @04:11PM (#14063130)
Could you be more specific? It doesn't seem like there's much room in the conventional definition of memory safety (e.g. https://en.wikipedia.org/wiki/Memory_safety ) for variation, other than, I suppose, putting conditions on when/which things are actually guaranteed.
Re:The reference D compiler is now open source by WalterBright on Friday April 07, 2017 @05:13PM (#14063543)
Re:The reference D compiler is now open source by dbaupp on Friday April 07, 2017 @05:49PM (#14063766)
Hm, I'm not sure I understand how that's answering my question. Neither of those seem to even define what the "memory safety" notion means for D so I can't compare. And, looking into it further, none of the links from https://dlang.org/spec/memory-safe-d.html define (or compare) what D means by "memory safety"/if it differs to the conventional definition.

I guess your original comment was maybe meaning how D enforces (the "normal" definition of) memory safety differs to Rust? In any case, I read over both of those, and, to me, they both seem to essentially be a slightly less general version of Rust's scheme (possibly independently invented), rather than something very different. I'm interested to hear how you think they specifically differ to Rust.

Re:The reference D compiler is now open source by WalterBright on Friday April 07, 2017 @07:33PM (#14064274)
Memory safety in D is the usual definition - making memory corruption impossible.

There's no notion of "borrowing" in D nor any notion of "only one mutable access at a time".

Re:The reference D compiler is now open source by dbaupp on Saturday April 08, 2017 @02:40AM (#14065670)
By not having borrowing, do you mean the compiler doesn't stop one from mutating something that has a dependent scoped pointer pointing into it? How does D avoid dangling pointers for that?
Re:The reference D compiler is now open source by leshow on Saturday April 08, 2017 @12:14PM (#14067615)
It has a GC, I think.
Re:The reference D compiler is now open source by floatboth on Friday April 07, 2017 @02:31PM (#14062312)
> a relatively slow compiler

Well… it's relatively slow in release mode, but C++ compilers can often be slower. GHC is always slower :D

Re:The reference D compiler is now open source by CyberShadow on Friday April 07, 2017 @04:25PM (#14063219)
For some reference: DMD (the reference D compiler this thread is about), which consists of about 160000 lines of D code and 100000 more lines of C++ code, compiles in a bit over 3 seconds on my machine.
Re:The reference D compiler is now open source by WalterBright on Friday April 07, 2017 @07:41PM (#14064306)
Counting lines of code per second for C++ can be tricky, because the .h files get compiled over and over for each project. The DMD front end is compiled with one command, and each D source file is read/parsed/compiled only once.
Re:The reference D compiler is now open source by deaddodo on Friday April 07, 2017 @03:26PM (#14062790)
If you plan to learn a systems language for systems reasons (embedded development, compilers, low-level interfaces, etc) then Rust is the pick. No GC and the option to have no runtime (or a very small one, by default).

In other words, if you're looking to replace/supplement your C use: go Rust. If you're looking to replace/supplement your C++ use: probably go D, with some caveats towards actual use.

Note: I like both languages, but do prefer Rust for my personal use cases.

Re:The reference D compiler is now open source by steveklabnik on Friday April 07, 2017 @03:34PM (#14062859)
> (or a very small one, by default).

Some details here... every language other than assembly languages has some amount of runtime. This is Rust's: https://github.com/rust-lang/rust/blob/master/src/libstd/rt.... (as you allude to, many people refer to this amount of runtime as "no runtime" since it's very, very small.)

You could also consider some other things as part of a runtime; for example, by default on most platforms, Rust includes jemalloc. That can be removed, though not in stable Rust. Same with the standard library code, which can be removed for libraries, but not binaries just yet (due to a small technicality regarding the stability of some attributes.) Things that do this are likely to use some of Rust's other unstable features at the moment, so it's not generally a burden to do so, and those interfaces rarely change right now.

Re:The reference D compiler is now open source by vonmoltke on Friday April 07, 2017 @06:54PM (#14064074)
> every language other than assembly languages has some amount of runtime

How are you defining "runtime" such that this statement is correct?

Re:The reference D compiler is now open source by steveklabnik on Friday April 07, 2017 @07:13PM (#14064159)
What language are you thinking of that I'm missing? C has crt.0, for example.
Re:The reference D compiler is now open source by vonmoltke on Saturday April 08, 2017 @09:06AM (#14066715)
crt0 does things you would need to do manually in hand-coded assembly anyway. It is no different than manually creating a library of assembly subroutines that you inject into your code, at which point assembly technically has a runtime as well.

I see a distinction between a runtime library and a language runtime. The former is just a set of convenient routines for interacting with a specific platform. The latter is something that runs alongside your program to facilitate its execution, like a JIT or an interpreter.

Re:The reference D compiler is now open source by steveklabnik on Sunday April 09, 2017 @10:14AM (#14072386)
> at which point assembly technically has a runtime as well.

The key is, you are doing the injecting in asm.

Anyway, your distinction is pretty solid. Most people just use "runtime" to mean either one, and then rely on the context to disambiguate.

Re:The reference D compiler is now open source by klickverbot on Friday April 07, 2017 @09:18PM (#14064694)
Do you depend on the C runtime as well?
Re:The reference D compiler is now open source by steveklabnik on Saturday April 08, 2017 @01:10AM (#14065425)
IIRC no; basically the same thing is done though, just not as a dependency.
Re:The reference D compiler is now open source by Keyframe on Friday April 07, 2017 @02:00PM (#14062015)
D2, as a language, favourably. As an ecosystem, not favourably.

I REALLY ENJOYED D1, REALLY! (CAPS 11). D1 was like C, but better. It was heavenly. D2 is like C++, but better. Not my cup of tea though. Give it a spin for a day or two. I'm sure you'll like it if you like C++. Trouble arises, as with all niche languages, when you try to move to production and you have to source libraries and/or support.

Re:The reference D compiler is now open source by jacques_chirac on Friday April 07, 2017 @01:38PM (#14061779)
Favorably. Pick D.
Re:The reference D compiler is now open source by kaleidic on Friday April 07, 2017 @04:59PM (#14063464)
What kind of support problems did you have? I found the opposite. It was possible to get such a good level of programmer from the community I was better off having found them letting them go work for the D Foundation on rebuilding compile time function execution. Libraries - meaning a bit more work to have idiomatic style use, but if you write D as C, that's usually okay - and wrapping is a one time cost.

I would say that I have found it to be more of a problem on Windows. That'd because the general C/C++ package management solution on Windows is quite... Old fashioned, and that's not always a D specific problem.

Re:The reference D compiler is now open source by pcwalton on Friday April 07, 2017 @02:12PM (#14062135)
Can you elaborate?