int i = f(3); // evaluate at run time
enum i = f(3); // evaluate at compile time
enum res = toggle_gpio(FRONT_LED_2);
Then, suppose we have all these requirements
- We want a separate compilation model where the code which is calling f(3) at either compile or run time knows nothing about how f is defined, only how it's declared.
- We want not to include the compiled image of f in the target program, if if is only called at compile time, only if at least one call to it somewhere in the program is staged to run-time.
- We want to make sure that if f is called at compile time, nobody can change its definition to accidentally call for run-time semantics like toggle_gpio(FRONT_LED_2).
and that's how we probably arrive at a declarative mechanism like constexpr that goes on the function, rather than trying to orchestrate thing remotely/indirectly by placing a call to the function into a compile-time context.
Not including functions never called for runtime execution in the runtime is a normal compiler optimization. No user input is necessary.
If the function's semantics change so it is no longer runnable at compile time, the compiler will let you know when you try it. If you want to force the detection, just call it at compile time with dummy declaration.
> that's how we probably arrive at a declarative mechanism like constexpr that goes on the function, rather than trying to orchestrate thing remotely/indirectly by placing a call to the function into a compile-time context.
This issue is raised now and then, and in 16 years of CTFE in D it has never ever been reported as causing an actual problem.
Great we have many good languages to choose from
This means, if a C library is available, D can use it.
(There are some exceptions. While ImportC will handle C preprocessor metaprogramming just fine, those metaprogramming macros won't be available to the D code, other than simple #define manifest constant declarations.)
Could it be because some of it, say, places a delegate, bound to a closure, into an immutable struct within a child thread's run function, and passes that to another thread via a send to be called?
Can't agree more. Such a powerful language but with an absolute meager ecosystem! No dogfooding.
GC and crashes are pretty common on long running high performance apps. They should just build things from the ground up using betterC.
So when I see comments like this that say "D also has X feature", I'm curious about why D didn't take off but Zig (likely) will.
In Zig's near category, there are other languages like Odin, Jai, Vlang, Rust, etc... For example, by the time Zig reaches 1.0, Jai (which is in Beta) might be publicly released, others languages might have added some "killer" features, and/or others in development might have also reached production quality.
Zig has quite a long way to go, and if anything may never achieve the popularity or widespread usage of D. Examples of the difficulty of programming languages rising through the ranks, is Nim and Crystal. Both Nim (2008) after 14 years and Crystal (2014), have not even made it into the top 50.
And no Vlang is not a programming language, it's a scam and should be treated as one.
Stop deluding yourself. If we take out TIOBE, and use the IEEE's Top Programming Languages of 2022 (https://spectrum.ieee.org/top-programming-languages-2022), D is still recognized (even higher than the 30s) while Zig, Nim, or Crystal do not even make their chart. That's the reality.
> Vlang...
To begin with, you are a known troll (with possibly multiple troll accounts at HN) that has spent over an year engaging in slander and lies about the language almost any time it's mentioned.
Looks like you are the creator or involved with some other language that can't get as much support or popularity. It would be advisable for you to stop being obsessed with Vlang. How many years of your life will you waste on such childish antics? You would be better off focusing on making your programming language better, if it's not already too late and its a failure.
As for pushing this bold face lie that the language is a "scam", that's both ludicrous and easily proven false. Vlang has many hundreds of code examples and projects. Easily found at:
1) Vlang examples on GitHub (https://github.com/vlang/V/tree/master/examples).
2) Vlang at Rosetta Code (https://rosettacode.org/wiki/Category:Vlang).
3) Awesome Vlang at GitHub (https://github.com/vlang/awesome-v).
The "scam" is you tricking yourself into thinking that your troll tactics are working. Instead, your continual and unnecessary defamation is making what you are even more obvious.
And for what, exactly? Your audience is the same audience you're claiming told Andrew it was a dumb idea, you're basically calling your audience dumb.
I choose to par-phrase him, and had no idea that this would get anyone this upset and lead to me being accused of being a liar. How much of experts these people where I cannot vouch for. Maybe they were second rate language designers or maybe Kelley referred to older papers advising against it.
Either way I cannot find the exact video anywhere where he made this remark despite looking through several. Does it really matter? Can you say for a fact that no language designer expert ever told Andrew Kelley this? I am not claim this is the general opinion among designers, only that this is what Andrew got told.
If anyone has the correct quote, I'll be happy to update the article.
You're an untrustworthy narrator and I stopped reading because I have better things to do than to submit myself to manipulation.
> Either way I cannot find the exact video anywhere where he made this remark despite looking through several. Does it really matter?
Gee, I dunno. Does it matter if I trust the person telling me something? Well shucks, I guess not.
This behavior is nothing short of antisocial. You're not attacking ideas here you're attacking a person.
Having a little grace for people goes a long way.
Or you know, the post is about more than that part, and whether those that dismissed the idea were experts (say SPJ or Hejlsberg) or just some programmers that dabble in compilers and had strong opinions, is irrelevant.
Of course by "stopping reading" one would never get to understand whether that's the case or not - god forbid any otherwise good article/post/book was frontloaded with something inaccurate...
> I was writing that intro based on recalling Andrew Kelley stating in a video
When you are writing for a technical crowd, it is good to be prepared with references and evidence. Unsourced information is likely to be questioned. Unsourced information making dubious claims that go against half a century of computer programming and practice is even more likely to be questioned.
> Can you say for a fact that no language designer expert ever told Andrew Kelley this?
That's not a valid counter-argument. Can you say for a fact that Russell's teapot does not exist? That does not make the existence of Russell's teapot any more likely! The burden of proof lies on you to show that the bold claim you made in your article holds up against scrutiny.
Making this into a huge topic that requires evidence and citations seems a bit over the top. I would say you guys don’t know how journalism works. You equate article writing with scientific journals and papers. They are not the same thing.
I think what is at the core is getting the language description correct.
> There are a lot of people in this thread saying that I said stuff I didn't say. It's maddening.
I have done it in the past and what I learned is that it can give a temporary boost of endorphins, but it's very short-lived and it doesn't really help anyone in the long term, including yourself.
I will try to lookup this video when I have time. I was a bit surprised that this remark was a such a turn-off. Had I known, I should have looked up the video to get the exact quote.
It was just meant as a bit of a funny dramatic entry to discussing an unusual language design choice for a statically type language. Sure we are seeing more things like const expressions in C++ today, but at the time this wasn't very common.
People can dismiss language ideas and concepts that "have been known and used for many decades" just the same.
For a trivial example: significant whitespace.
So, e.g. "Lisp has had that since forever" is not a guarantee people would be OK with it, unless they also like Lisp - not to mention they might not even know about it in Lisp.
As for C++, many people consider templates - at least as implemented - a bad idea. So some people knowing that it existed in C++ already, and considering it a bad idea to have in Zig is not contradictory.
The phrase "They said it couldn't be done!" is an evocative proxy for technological novelty. If the experts are not representative, the claim of novelty will be misleading.
Well, it did create a big mess and overhead in C++ for one (not to mention adding another turing-complete language on top of it)...
This was actually a schism in Lisp and why FEXPRs died out. CS theory wanted to compile things for speed and FEXPRs very much didn't fit into that. This is why macros took over--they were very much targeted at compile time.
The late John Shutt's thesis and work on the language he called Kernel goes into quite a bit of detail and the history about it: https://web.cs.wpi.edu/~jshutt/kernel.html
Sadly, Shutt never really "completed" the Kernel language--the language is very interesting but is extremely slow and unwieldy in the descriptions he gave. It generates a lot of garbage and extra "environments"; it resists compilation and memoization; it sometimes seems to go out of its way to deal with things that make life very difficult (circular and infinite lists).
A couple of good passes of optimization might have produced something very interesting.
I was personally attracted to it for very small embedded devices (<16K RAM/16K Flash) as it delgates a lot of macrology to interpretation--sadly I just couldn't get the footprint down far enough given all the garbage and environments the language generates. I suspect it's possible, but it would take someone who knew the landscape intimately to figure out what needed to be dropped.
> infinite lists
Sounds like an early attempt at Haskell. Haskell is lazy by default so you can write your own "if" as a function and it will be just as short circuiting as the built in if. And infinite lists show up in basic Haskell tutorials.
However Haskell combines this laziness with immutability, which is an easier combination to reason about than laziness with mutable side effects.
Compiler technology for lazy languages is definitely impressive, but it's still the case that if you want to squeeze the last bit of performance out of Haskell code it does usually require manually forcing eager evaluation.
See: "Special Forms in Lisp" by Pitman https://dl.acm.org/doi/pdf/10.1145/800087.802804
Shutt's thesis does a good job talking about the history.
Circle, mentioned elsewhere, is a C++ dialect with fully working compile time metaprogramming.
And the error handling is even only 10% working. clang is fine, gcc is broken for years.
See AIM-57: http://publications.csail.mit.edu/ai/browse/0000browse.shtml
Does there have to be? Is there a language that has absorbed its build system into it?
I agree with this, but there may still be times when it is needed.
I'm working on a system where users can run a build in an interpreter that kills the build if it tries to do anything they don't allow. An important and trusted project that needs a special permission will probably be allowed to do that, but random packages? Not at all.
Saying the compiler can't run user code and make system calls while compiling is is like plugging a whole the size of a bus with your pinky. You prevented absolutely nothing. You just make devs jump through hoops (external tools in their build) to do the things they need to do and in doing so force them to add dependencies which expand their surface area of attacks.
Maybe a command-line switch to say which files can execute code at compile time but still, having worked on a lisp system that ran code and made system calls at compile time it was a huge time saver. Example:
constexpr max_buffer_size = getSizeOfLargestFile("assets/*.gltf");
enum Modes = enumerateSupportedModes("modes/*.el");So at least some IDEs are more cautious about this these days.
However, I think that if an interpreter does the execution, the interpreter can check every "instruction" that is executed. If every instruction is checked, how could that be evaded?
Honest question because I can't see how, and I need to.
The context for this is an interpreter that doesn't allow direct syscalls; only syscalls through the interpreter.
It seems even D has that because dynamic allocation could call `mmap()`, and you mentioned that D allows comptime dynamic allocation. So I'm confused.
Whether this is desirable is dubious. But fundamentally it's not any more dangerous than Mara's nightly_crimes! Rust proc macro (which during compilation replaces your running compiler with a compiler that believes it is compiling the standard library and thus allows non-stable "nightly" features even though you aren't running a nightly build, then casually alters the compiler output to say that this was fine and there's nothing to worry about), or any number of other tricks which result from being powerful enough at compile time.
I doubt that D is as powerful as people would want and yet manages to ensure this can't happen. There are Rust people thinking about WASM sandboxing for this, but it's tricky.
with the GC, which is safe
And yes, this is a trillion dollar problem. I may be just some bloke with an idea, trying to make it happen. However, I have the free time to try, so why not?
In a language like D you could just execute the same system calls in the executable that was compiled. The assumption that you compile something and then not run it doesn't make this any safer, does it?
Or is it still for in-house use only?
reportedly his game "The Witness" was written in it. He has a series of video's where he explores what the language will look like as he's developing it, and has claimed he'll open source it once he feels it's ready. He wants to maintain control until he feels it's no longer half-baked.
IIRC, he demonstrates things like a music player running during compile time. It's a core tenet of his language.
Probably the best place to see it in it's current form is his development streams on Twitch: https://www.twitch.tv/j_blow. Right now the most active streamer besides Jon is Raphael, who does a lot of work on the Jai compiler: https://www.twitch.tv/raphael_luba. Sadly, the other people I know who stream their work with it haven't been streaming much lately.
No that was still written in C++. The new game that he's working on is written in Jai.
At the moment you can't do comptime allocations, so everything has to exist on the stack, but you can, for example, embed a file like a config or dataset, and run it thru a parser to turn it into some data structure, which is something you might otherwise work into the build system, running python/perl scripts to generate temporary code files, but now you can cut out the extra dependency by having that code generation in the same language.
Zig's use of comptime is fairly revolutionary in a c-like language. Compared to the shenanigans required by c++, the entire language* is (should be) available at comptime. Compared to the backtick/quote/macro magic available in lisp... the awesomeness isn't that it's comptime, the awesomeness is that it's type safe.
But, I'd say it isn't even the use of comptime that's particularly noteworthy, it's the lack of pretty much every feature that isn't comptime. This wouldn't be a good idea without comptime being as robust and featureful as it is.
* nope, see puffoflogic's reply
The D programming language popularized it in 2006 or 2007. Here's it in action:
int square(int x) { return x * x; }
int[square(3)] a; // declare array `a` of 9 ints
int square(int x) { return x * x; }
_Static_assert(square(3) == 9, "Error, Will Robinson!");
int[4] squares1 = [0, 1, 4, 9]; // doing it by hand
int[4] squares2 = () {
int[4] a;
foreach (i; 0 .. 4)
a[i] = square(i);
return a;
} (); // doing it with CTFE
I actually include myself in that camp, so thanks for taking the stage :D
https://www.wildernesslabs.co/
https://nikolayk.medium.com/getting-started-with-unity-dots-...
Had Remedy experiment worked out, maybe DOTS would be using D instead of having their own compiler infrastructure for HPC#.
Nor Java even,
https://www.ptc.com/en/products/developer-tools/perc
https://www.aicas.com/wp/products-services/jamaicavm/
The main reasons are not having a big name company that would push D no matter what, and lack of focus where D should aim for.
You can ensure your D code doesn't use the GC by adding the `@nogc` attribute.
But really, all this concern about the GC is misplaced. It's just another tool available. You can use it in D, or use RAII, or your own allocation system. It's your choice as a D programmer.
I am giving this analogy since Walter has a degree in mechanical engineering and he has correctly made the GC a default in D but not mandatory, but the choice somehow is not popular in the software industry.
int squares1[4] = {0, 1, 4, 9}; // doing it by hand
std::array squares2 = [] {
std::array<int, 4> a;
for (int i = 0; i < 4; i++)
a[i] = i * i;
return a;
} (); // doing it with CTF
string cat(string s) {
return "int " ~ s ~ ";";
}
int test() {
mixin(cat("i"));
return i + 3;
}So what if D wins at code golf against C++?
What matters is being good enough for most compile time use cases, while enjoying the large ecosystem and IDE tooling, perfect is the enemy of good.
Indeed C# and C++ don't give me all nice features from D, when doing language comparison tables, D wins on that table listing.
Yet they give me their library ecosystem, graphical IDEs, and first day avail on any major company SDK.
I will take that productivity win over the few features that the D language happens to have.
Why is Andrei Alexandrescu helping to improve C++ at NVidia instead of using D, when the language is so great?
Ecosystem, that is why.
const std = @import("std");
export fn square(x: c_long) c_longlong {
comptime std.debug.assert(@sizeOf(c_longlong) > @sizeOf(c_long));
return @as(c_longlong, x) * x;
}
Yes:
c_longlong square(c_long x) {
static assert(c_longlong.sizeof > c_long.sizeof);
return cast(c_longlong)x * x;
} template MakePtrType(T) { alias MakePtrType = T*; }
MakePtrType!int p;
pragma(msg, "p is type ", typeof(p));
dmd -c test2.d
p is type int*
Edit: So the main difference seems to be that types are first class citizens in Zig, you can write a normal comptime function that returns a type and either use it to do more metaprogramming or everywhere you would declare a type for a function or variable. I am not aware of any static typed languages that lets you do this with normal language syntax.
C++ does it with templates, but in Zig you can do what C++ templates can but with if statements and loops.
Please show me an example where it differs. D's metaprogramming is very good at constructing types. A member of the D community tried very hard to implement type functions, but it turned out to be redundant with D's existing capability.
Not revolutionary. Before Zig came out, which was 2016, there was Jonathan Blow talking about and arguably repopularizing it. Lots of people (who knows how many developers) and languages were influenced by him too.
2014, Jonathan Blow, who made lots of videos and gave lots of talks, before going on to officially promote Jai (2016). And it appears he was playing around with prototypes much earlier and talking about it (from 2014).
https://youtu.be/UTqZNujQOlA (Demo: Base language, compile-time execution)
I might phrase that as "statically-typed language". Most (if not all) of the dynamically-typed languages are technically programs that run at the only "compile time" there is and can do arbitrarily whacky things as a result. It is generally a good idea not to think of them that way. I've removed quite a few top-level Perl statements in my time and enforced rules that a module really should be nothing but declarations of constants and subroutines unless there's an absolute emergency. But dynamically typed languages will generally let you read some files off a disk, query a database schema, and hit an HTTP API to build a "module" if you want to.
And in this case the entire language really is available. The limitations tend to come with the increasingly fragile order of operations the modules impose on each other, rather than technical capability.
I don't say this to slag on Zig. Increasing the range of what static languages can do is a legitimately interesting topic. This is to shed light on the general space of programming langauges.
Sometimes it's legitimately an innovation just to put it into a standard library from the very beginning. I've discussed on HN a couple of times how useful it was for Go to ship with an "io.Reader" definition for "things that emit bytestreams" in the standard library. It has very little to do with language features; many other languages could theoretically have done it. Many languages have a sort of half-usable "file" abstraction, but you can never quite tell what calls will work on which type of thing. Having a single concrete interface in the standard library from day one of the Go language drove the entire ecosystem in a single coherent direction, though, and in Go, you can pretty much count on that interface and can nest it quite deeply without hitting a corner case.
I expect it may be something like that for Zig... it's not that nobody has ever integrated code generation at the compile step for a static language before, it is merely that I'm not sure anyone's ever pushed it out like from the very beginning at this level. I may be wrong. I can come up with several examples of much more dynamic languages doing it. I'm sure out there in the world you can combine Lisp macros with some static type system for the same language. But that would be a bolt-on, not something that was in the standard lib from day one.
Thinking of languages not just in terms of features, but in terms of what community the standard library affords is probably the next major frontier in language design over the next 20-30 years. Yeah, I hope that we still see some sort of revolutionary language that solves all our problems, but I expect to see a lot of languages that don't necessarily have any new features per se but are just better standard libraries from day one.
But yeah, dynamically typed languages have been doing this for decades, particularly prevalent in Perl circles.
It it is not revolutionary in the context of such languages. Out of the mainstream but not revolutionary.
I haven't used Zig (yet, maybe one day), but does anyone know if there's a difference between Haxe macros and Zig comptime? AFAIK Circle also offers something similar for C++ (https://www.circle-lang.org/)
This makes it less expressive, but it also means it is easier to learn and use. (It's a lot like C++'s constexpr)
The interesting part comes from recognizing that by making constructs like structs first-class-values in comptime, you can achieve most of the common things that templates, macros, conditional compilation, etc give you, but with a much simpler feature (which is just running code at compile time)
Like with Zig, you are limited mostly by what is available in the environment at macro expansion time; unlike Zig, you can do things that generate allocations.
> but with a much simpler feature (which is just running code at compile time)
Common Lisp does that with the EVAL-WHEN construct, which allows to run arbitrary code at compile time.
That’s exactly what Haxe macros are. It’s regular Haxe code that runs during compile, and it has access to all Haxe language features and the standard library.
The only special thing about them is that they can directly manipulate the AST before it gets passed on to the compiler for final code generation.
In a sense, they’re like “shaders” for the compiler.
https://github.com/ziglang/zig/issues/1291
This is an open issue from 2018 that I opened that specifically acknowledges this as a valuable use case that should work.
Don't let puffoflogic put words in my mouth. They are a troll account.
One major use is to create strings at compile time, which are then fed back into the compiler as D code. I.e. metaprogramming.
[1]: https://ziglang.org/documentation/master/std/#root;fmt.compt... [2]: https://ziglang.org/documentation/master/#compileError
Fixed-size allocators are not composable. You cannot write any kind of comptime routine which calls further routines which need to do allocation, and how much allocation they do is based on arguments to your routine - except by forcing your caller to calculate how much allocation needs to be done. But that leaks your implementation details to your caller, largely invalidating the point of making some kind of comptime routine in the first place. Therefore comptime can be used as a toy, or in C-style language-enforced NIH mode (where you [re]write every single routine you need yourself, or at least know the intimate implementation details of all of them), but not in a serious programming stance.
Sure, if it's all code you're writing yourself then it's doable to avoid using an allocator. If you want to reuse existing code that does expect an allocator, however, then being able to use a FBA is handy.
Runtime allocators also have a limit to how much memory you can allocate. It's usually higher, but there's still a limit. Your computer doesn't have an infinite amount of memory.
Overall I am not the right guy for this conversation because I am too used to never calling mmap at runtime either.
I want to give zig a fair assessment so yes, this is absolutely right. This is why I have elsewhere said that zig is basically two separate but superficially similar languages. You can probably accomplish just about anything in comptime zig, if you're willing to completely reimplement it from the core language up. The limitations I mention come in if you try to write code to be used in either comptime or runtime zig. So comptime zig has no growable arrays, no dictionaries, no string searching, no regexes, etc. etc. until these are reimplemented specifically for comptime zig. Mostly this is just sad, that a decent idea was implemented with such a terrible downside that would be so easy to fix: just add allocators. Just. Add. Allocators. But no, the zig team has explicitly rejected this.
I'm torn on this. To me, comptime is the C preprocessor but sane. Anything you can do above that is bonus. I don't need comptime to be totally Turing complete.
And, I do NOT want macros in the language. Macros are a bottomless well of suck. Lisps/Schemes still tie themselves into knots with macro hygeine. Rust's procmacros are terrible and debugging them is worse--just try figuring out what some intermediate expansion did.
The point of Zig is to NOT be Rust/C++/etc. The point is to be C with a bunch of the sharp edges and corners filed off. It's not for everybody. If you want web services, use Go. ML/AI--Python for ecosystem. Rust for strong safety guarantees.
Zig has some nice things while still remaining relatively small. It, like most modern languages, understands that ecosystem is important. It does a better job than almost everybody at cross compiling. comptime gets you a lot of the benefits of templates while dodging lots of the suckage.
However, at some point, you have to declare "Stop, that isn't going to be part of the language" or you wind up with the ungodly mess that is C++.
It's not, it's just fairly insecure. comptime is known forever, in static and dynamic languages. Better languages do know about the security and usage implications and do something about it.
So fopen, print and all IO or OS functions should be avoided, or even warned. It might be useful in certain edge-cases, but in my 20 years of compile-time programming, it's almost ever a bug, not a feature.
Example: the BEGIN block is executed during the compile phase as the parser encounters it. The code injected in BEGIN blocks can affect how the rest of the file is parsed.
$ perl -E 'exit; BEGIN { say "Hello" }'
Hello
$ perl -c -E 'exit 1; BEGIN { say "Hello" }'; echo $?
Hello
-e syntax OK
0
* https://perldoc.perl.org/perlmod#BEGIN,-UNITCHECK,-CHECK,-IN...