News for hackers. Stuff that matters.
Re:Adding ANSI C11 C compiler to D so it can import and comp... by txdv on Monday May 10, 2021 @02:05AM (#27103170)
Adding the ability to just read header files and all the definitions in them without having to translate.

I saw this addition in zig and thought that it was amazing, a lot of languages have this need to create definitions in their own language, C# with [DllImport](https://docs.microsoft.com/en-us/dotnet/api/system.runtime.i...), [crystal with lib](https://crystal-lang.org/reference/syntax_and_semantics/c_bi...), I think since C is so ubiquitous and being able to just consume header files just removes so much boilerplate creation that is easy to get wrong.

zig also decided to just call an existing compiler (clang) and if you do zig cc --version then zig thinks it is clang, because the zig compiler just calls into clang main. D has gone with the route of calling gcc.

Re:Adding ANSI C11 C compiler to D so it can import and comp... by WalterBright on Monday May 10, 2021 @05:10AM (#27104100)
> D has gone with the route of calling gcc.

Not exactly. D is going the route of actually implementing a C compiler in the D compiler. Here's the C parser:

https://github.com/dlang/dmd/pull/12507/files#diff-2fc171ca9...

You could argue that writing a C compiler instead of using libclang is an indicator of brain-damage, but I lay out my reasons here:

https://digitalmars.com/d/archives/digitalmars/D/Add_ImportC...

And besides, C is such a simple language it's kinda fun.

Re:Adding ANSI C11 C compiler to D so it can import and comp... by AndyKelley on Monday May 10, 2021 @01:50PM (#27108768)
> 9. Without a C compiler, we're stuck with, wedded to, and beholden to libclang.

> I wouldn't be surprised that the eventual cost of adapting ourselves to

> libclang will exceed the cost of doing our own C compiler.

This is a really insightful point. I had to learn this the hard way :)

We might follow your lead on this, as we have done with so many other great ideas implemented in D.

Ironically, Vexu started from the other side as you, with the preprocessor mostly done, but the backend left to-do: https://github.com/Vexu/arocc

One thing that might make libclang worth the cost, however, is its ability to compile C++ code as well. On Zig's end of things, all we have to do is provide libcxx, libcxxabi, libunwind, compiler-rt, and linking, and then libclang is really pulling a lot of weight by compiling C++ code into object files. Sadly this ability is just too useful in practice to ignore. For example, LLVM itself is C++ so if Zig wants to be able to bootstrap itself, it needs this capability.

Still, I think your maneuver here is the best long-term approach to tackle this problem, and I imagine as time goes on we'll start to migrate towards D's solution here. Maybe someday the Zig distribution that does not have LLVM extensions enabled will be the more popular one.

I'll be watching the evolution of this new feature in D with great interest!

Re:Adding ANSI C11 C compiler to D so it can import and comp... by WalterBright on Monday May 10, 2021 @02:57PM (#27109502)
I agree that this technique of writing my own C compiler isn't going to work so well with C++:

1. Even though I have written a C++ compiler, it's a C++98 compiler and C++, like the Winchester House, accretes new features constantly, meaning keeping it up to date is impractical.

2. C++ semantics are different enough from D that it is impractical to mechanically map C++ to D. You can do bits and pieces, and dpp does that, but seamlessly doing the bulk of it isn't going to work. (C can be mapped nearly 100% to D.)

dpp uses libclang to try and map what it can to D.

Re:Adding ANSI C11 C compiler to D so it can import and comp... by beeforpork on Monday May 10, 2021 @05:30AM (#27104220)
So how extremely could I exploit this? Could I write a program entirely in C, compile it with the D compiler, and use D's unit test framework? That would be insane (very good insane, obviously).
Re:Adding ANSI C11 C compiler to D so it can import and comp... by WalterBright on Monday May 10, 2021 @05:40AM (#27104284)
That is a great question. The point of ImportC is to import the declarations from a C file. But it turns out you can simply give the C file on the command line, and it'll compile it and make an executable:

    echo hello.c
    int printf(const char*,...);
    int main() { printf("hello world\n"); }

    dmd hello.c

    ./hello
    hello world
But then there's the question "should D features be allowed in the C code?" This is a very subversive question. Is it really C if that is allowed? Already, the C code allows for Compile Time Function Execution, which comes for free by passing the C code through the D front end. You can see it in action in the test cases in this file:

https://github.com/dlang/dmd/pull/12507/files#diff-b94776a02...

Scroll down a bit and you'll see it executes C functions at compile time. I decided to flirt with heresy and leave this in, because it sure makes adding tests convenient!

I'm less sure about adding unittests to the C compiler. Certainly, you can always add the unittests to the D code that is importing the C code. This will probably hold the charges of abomination to a minimum.

Re:Adding ANSI C11 C compiler to D so it can import and comp... by crazypython on Monday May 10, 2021 @07:46AM (#27104877)
> Already, the C code allows for Compile Time Function Execution, which comes for free by passing the C code through the D front end.

Oh my good gosh. D and C could be seamless together. This is great.

> I decided to flirt with heresy and leave this in

I'd love to see more C language extensions in ImportC.

Re:Adding ANSI C11 C compiler to D so it can import and comp... by agumonkey on Monday May 10, 2021 @12:17PM (#27107684)
Do you produce a D ~object from c source which is then processed by dmd ?
Re:Adding ANSI C11 C compiler to D so it can import and comp... by mhh__ on Monday May 10, 2021 @12:54PM (#27108104)
It produces a D AST from C syntax.
Re:Adding ANSI C11 C compiler to D so it can import and comp... by mistrial9 on Monday May 10, 2021 @09:33AM (#27105801)
congrats! C is simple sure, in some ways.. but it is the complete standard library that makes it behave like "C" for writing programs. This was by design of course
Re:Adding ANSI C11 C compiler to D so it can import and comp... by tomcam on Wednesday May 12, 2021 @03:08AM (#27127576)
You are a beast
Re:Adding ANSI C11 C compiler to D so it can import and comp... by _old_dude_ on Monday May 10, 2021 @03:21AM (#27103533)
yes, Java is adding jextract [1] as a replacement for JNI (Java Native Interface) which directly consumes header files using clang and produces a jar wrapping the native calls.

It's not bolted in the compiler so less nice than Zig but the jar produced by jextract is not ABI dependent so more portable. The ABI dependent code is generated by the JIT using the the Foreign Linker API (the equivalent of C# DllImport) at runtime.

[1] https://inside.java/2020/10/06/jextract/

Re:Adding ANSI C11 C compiler to D so it can import and comp... by pjmlp on Monday May 10, 2021 @04:16AM (#27103787)
Actually the equivalent of @dll.import from J++, funny how things turn out.

https://docs.microsoft.com/en-us/previous-versions/visualstu...

Re:Adding ANSI C11 C compiler to D so it can import and comp... by MrBuddyCasino on Monday May 10, 2021 @05:15AM (#27104123)
That code also contains a nice C parser that is handy, backed by LLVM. I recently switched from JNA to Panama for a project, and so far no regrets, other than having to mess with the Java module system.
Re:Adding ANSI C11 C compiler to D so it can import and comp... by pjmlp on Monday May 10, 2021 @07:41AM (#27104836)
It is already stable enough?
Re:Adding ANSI C11 C compiler to D so it can import and comp... by MrBuddyCasino on Tuesday May 11, 2021 @03:10AM (#27115128)
Too early to tell. It clearly has received more care & attention so far than all the alternatives, so I’m not too worried about it being in „incubator“ status. The project lead knows his shit.
Re:Adding ANSI C11 C compiler to D so it can import and comp... by dleslie on Monday May 10, 2021 @02:18AM (#27103241)
Scheme is a language where I wish this was part of the spec. So many Schemes are intended to be embedded or extended with C, and every one of them has a bespoke way of declaring C bindings and martialling data between C and Scheme.
Re:Adding ANSI C11 C compiler to D so it can import and comp... by shakna on Monday May 10, 2021 @02:46AM (#27103356)
There are ongoing efforts to stabilise a SRFI (like SRFI198 [0]), which would be the first step towards this. (First get the SRFI, then once that's stable and well implemented, it can be considered for the next spec).

Unfortunately there's a lot of disagreement between the bigger players as to how this should be handled:

> > 1) The authors appear to be deadlocked

> No need to soften it with an "appear", we are intractably deadlocked.

[0] https://srfi.schemers.org/srfi-198/srfi-198.html

Re:Adding ANSI C11 C compiler to D so it can import and comp... by froh on Monday May 10, 2021 @02:37AM (#27103323)
I agree.

to bystanders, odd autocorrect: it's marshalling. marshalling data. although fight and war are indeed also appropriate associations for binary data exchange (struct conversions) between languages.

Re:Adding ANSI C11 C compiler to D so it can import and comp... by dmurray on Monday May 10, 2021 @06:56AM (#27104598)
TIL martial and marshal are unrelated etymologically, though marshalling data is by analogy with marshalling troops, and related to the military office.

Marshal = mariscalcus = "horse servant", compare mare and seneschal. Martial = from Mars, god of war.

https://en.wiktionary.org/wiki/marshal

Re:Adding ANSI C11 C compiler to D so it can import and comp... by froh on Monday May 10, 2021 @09:52AM (#27105973)
TIL: marshalling == ceremonial arrangement and management of a gathering

these CS gals of olde really had a hand for naming :-)

Re:Adding ANSI C11 C compiler to D so it can import and comp... by watersb on Monday May 10, 2021 @04:29AM (#27103853)
I often court martial my data.

Not mocking the spelling of the previous post; it's a strange term anyway and in this instance either spelling works.

Re:Adding ANSI C11 C compiler to D so it can import and comp... by TazeTSchnitzel on Monday May 10, 2021 @03:05AM (#27103434)
`zig cc` makes Zig into a more convenient C cross-compiler than upstream Clang!
Re:Adding ANSI C11 C compiler to D so it can import and comp... by viraptor on Monday May 10, 2021 @03:28AM (#27103572)
Why is it more convenient?
Re:Adding ANSI C11 C compiler to D so it can import and comp... by nickcw on Monday May 10, 2021 @03:40AM (#27103632)
Zig can cross compile and link libc on lots of different architectures out of the box, which is amazing given how much effort this takes to set up cross compilers normally.
Re:Adding ANSI C11 C compiler to D so it can import and comp... by slimsag on Monday May 10, 2021 @05:18AM (#27104148)
saurik's reply to your comment feels to me like the epitome of the seasoned C/C++ developer _who is able and willing_ to put up with the tooling setup pains that make C/C++ a non-starter to the "average" developer today.

You can absolutely set up a cross-compiling C/C++ toolchain today as he said - but you're going to have to find the right packages in your distro, deal with the MacOS linking issue, get a copy of the android NDK, make sure you're installing the right package versions, muck around with libc, etc. That doesn't make for a delightful start to a new weekend project.

It's often underestimated just how substantial "setup pains" are IMO. I suspect most people who have a strong distaste for C/C++ mostly despise the tooling and installation hell.

The novel part of what Zig, (and maybe Rust/Go to a lesser extent) are doing with cross-compilation is not 'making cross compilation possible' - it's having 'the best experience' out of the box, with zero installation pain.

Zig has a ton of issues and is super early stages still.

But shipping out-of-the-box with a Zig/C/C++ cross-compiler, with libc, static libc(musl), compiler-rt, libunwind, libcxx, etc. for ~50 cross-compilation targets, and solving the MacOS cross-compilation linking issue (with their customer linker, zld) is IMO a huge deal because it lowers the barrier to entry to systems programming quite substantially.

Re:Adding ANSI C11 C compiler to D so it can import and comp... by pjmlp on Monday May 10, 2021 @07:48AM (#27104897)
> saurik's reply to your comment feels to me like the epitome of the seasoned C/C++ developer _who is able and willing_ to put up with the tooling setup pains that make C/C++ a non-starter to the "average" developer today.

For me the best example I currently have of such attitude is how an internal WinDev group managed to push for getting C++/CX replaced with C++/WinRT without it being production ready.

After 4 years, still no Visual Studio support for IDL files, one has to manually copy/merge generated files into the Visual Studio project, no two way editing with XAML files, we get continuously told that it only will be fixed when ISO C++ gets compile time reflection.

Apparently they feel that is a productive experience and never came close to try out either C++ Builder or QtDesigner.

I guess they were still using raw C to deal with COM to consider this kind of downgrade as something positive.

Re:Adding ANSI C11 C compiler to D so it can import and comp... by saurik on Monday May 10, 2021 @04:50AM (#27103974)
FWIW, this isn't actually difficult: you just need a sysroot, and past that you can just use clang (maybe ironically, for any platform other than Apple ones, as there still isn't a good common linker for mach-O; like, this still works fine, but it is in fact very annoying--but not impossible--to compile a macOS binary from any platform other than macOS). To cross-compile to Linux you can just download a handful of rpm files (the kernel and libc header packages, as well as the gcc package for the CRT) from CentOS or deb files from Ubuntu and extract them to a folder, and for Windows you just need a handful of Msys2 packages (the same general set). That's really all you need: my current major project--Orchid--compiles for Linux, macOS, iOS, Android, and Windows, and does so using the Android NDK copy of clang (just because it is easy to install and easy to ensure us always "the same" on every machine: I do cross-compiled reproducible builds). As long as you are using C/C++ it really should be easy to cross-compile (and yes: this is saurik saying something not only good about clang but specifically saying they finally nailed this use case, which they really hadn't a while back when people kept touting this as a feature... they really did eventually nail it). (That said, I do appreciate that not having to download and extract those packages is certainly "more convenient" than doing anything at all, but I just want to point out that it really isn't that "much effort".)
Re:Adding ANSI C11 C compiler to D so it can import and comp... by uh_uh on Monday May 10, 2021 @09:08AM (#27105563)
This sounds a lot more difficult than using Zig where you just need to find the right build flag. Maybe it's easy once you already know which exact packages to get for each platform, where to put them and how to tell clang about them. But if you don't know this, researching this can easily take hours, at least for me.
Re:Adding ANSI C11 C compiler to D so it can import and comp... by AndyKelley on Monday May 10, 2021 @01:58PM (#27108872)
your solution would not support:

* specifying target cpu features (important!)

* features such as thread sanitizer support

* LTO that crosses the libc boundary

* targeting any glibc version

...all of which are supported out of the box by zig.

Re:Adding ANSI C11 C compiler to D so it can import and comp... by matu3ba on Monday May 10, 2021 @04:57AM (#27104012)
1. Do you have a blog post with exact instructions for this for some of the platforms? 2. Does this include targeting any gnu libc version (backwards compatible)? This would be very surprising to me.
Re:Adding ANSI C11 C compiler to D so it can import and comp... by p0nce on Monday May 10, 2021 @06:57AM (#27104602)
The macOS D compiler cross-compiles to iOS, x86_64, and arm64. It just ships with 3 standard libraries built for those arch.
Re:Adding ANSI C11 C compiler to D so it can import and comp... by nikki93 on Monday May 10, 2021 @02:16AM (#27103231)
LuaJIT was "good enough" at this that it worked pretty well for me -- mostly for automatically binding to headers from my own code where I was careful to avoid some of the potential gotchas. With a macro the header was stringified and sent to LuaJIT; could've also had a build step. eg: https://github.com/nikki93/cgame/blob/master/src/sprite.h#L9

An added benefit is you get ffi-reflect -- https://github.com/corsix/ffi-reflect -- over the resulting types / functions...

Re:Adding ANSI C11 C compiler to D so it can import and comp... by Rounin on Monday May 10, 2021 @04:36AM (#27103885)
Not that this is terribly relevant to the discussion, but back in the day, I did try compiling bzflag with zig cc and then with clang, and the first executable crashed, while the second worked.

Despite being told numerous compile-time options to use by different manuals and documents and people on IRC, nothing appeared to change the outcome. It might be due to some undefined behaviour in bzflag, of course, but it made zig cc a non-starter for C.

Re:Adding ANSI C11 C compiler to D so it can import and comp... by ifreund on Monday May 10, 2021 @04:55AM (#27103998)
This is due to zig enabling clangs UBSAN by default in unoptimized builds. This can be disabled with `-fno-sanitize=undefined` or by enabling optimizations with `-O2` or `-O3`.

https://github.com/ziglang/zig/issues/4830#issuecomment-6054...

Re:Adding ANSI C11 C compiler to D so it can import and comp... by Rounin on Monday May 10, 2021 @05:16AM (#27104128)
Well, as mentioned, I got a whole list of different compile-time options to try - no-this and no-that, but the executable still crashed in the end.
Re:Adding ANSI C11 C compiler to D so it can import and comp... by matu3ba on Monday May 10, 2021 @04:53AM (#27103988)
Please be specific with version numbers. Did you file an upstream bug already? Do you mean https://github.com/BZFlag-Dev/bzflag ? Thats c++ and must be compiled with zig c++.
Re:Adding ANSI C11 C compiler to D so it can import and comp... by Rounin on Monday May 10, 2021 @05:14AM (#27104117)
Ah, well, it must have been zig c++, then, necessarily. Was I planning to file an upstream bug? I was unaware. I did contact them about it, actually, but it wasn't solved at the time.
Re:Adding ANSI C11 C compiler to D so it can import and comp... by DrBazza on Monday May 10, 2021 @05:58AM (#27104356)
Apropos of nothing, but found out that the zig compiler can run C++: https://twitter.com/andy_kelley/status/1391468499676008451?s...