Jai allows compile time execution of system calls. We (the D team) considered this, and rejected it because we didn't want users to be worried that there'd be some malicious source code that could delete their file system if they just ran it through the compiler.

I'm wondering if the premise was that people compiling other people's code should be prioritized over "actual" users who develop their software using the compiler, who then would need to resort to external tools to do more complicated things, nullifying the protection anyway; or that it's expected that developers routinely include code they don't trust?

People expect the compile step to be trustworthy.

Sure, many people do, and some don't. I suppose the point I'm trying to make is that decisions like that make one consider what are the assumptions and target audience of a project. In this case, as a potential user, if I'm looking for a language that's a deadly tool for doing difficult things, intentional limitations feel almost patronizing.

Most build systems allow you to run arbitrary commands anyway.

But there is a clear distinction between the build system and the programming language. There is also usually much less code in the build system and it will be reviewed with the idea in mind that it could do something dangerous at build time. Reviewing every line of source code with compile time effects in mind could be very hard and sometimes the reviewer might easily miss the fact that something will happen at compile time, especially if it is hidden behind some api.

There is no important difference between running compile time code using language primitives, or compiling and then running a program as part of the build. You can review the build system, but you won't find the "evil" stuff in there.

> But there is a clear distinction between the build system and the programming language.

Does there have to be? Is there a language that has absorbed its build system into it?

Not GP.

I agree with this, but there may still be times when it is needed.

I'm working on a system where users can run a build in an interpreter that kills the build if it tries to do anything they don't allow. An important and trusted project that needs a special permission will probably be allowed to do that, but random packages? Not at all.

The trouble is, malicious people can be very clever at evading such protections. We didn't want to get into an arms race with them. I'm old enough to remember ActiveX.

The code being compiled, when run, could do anything. As for build time, nearly all the build systems that people use in languages like C, C++, Java, Node, etc can do nearly anything.

Saying the compiler can't run user code and make system calls while compiling is is like plugging a whole the size of a bus with your pinky. You prevented absolutely nothing. You just make devs jump through hoops (external tools in their build) to do the things they need to do and in doing so force them to add dependencies which expand their surface area of attacks.

Maybe a command-line switch to say which files can execute code at compile time but still, having worked on a lisp system that ran code and made system calls at compile time it was a huge time saver. Example:

    constexpr max_buffer_size = getSizeOfLargestFile("assets/*.gltf");

    enum Modes = enumerateSupportedModes("modes/*.el");

The problem is an IDE will start compiling code as soon as you open it just to look at it.

Jetbrains products have started prompting if you'd like to open a new project in "safe" mode that disables the build tooling until you've had time to evaluate the project.

So at least some IDEs are more cautious about this these days.

That seems true.

However, I think that if an interpreter does the execution, the interpreter can check every "instruction" that is executed. If every instruction is checked, how could that be evaded?

Honest question because I can't see how, and I need to.

The context for this is an interpreter that doesn't allow direct syscalls; only syscalls through the interpreter.

It seems even D has that because dynamic allocation could call `mmap()`, and you mentioned that D allows comptime dynamic allocation. So I'm confused.

Jai allows arbitrary syscalls at compile time, AFAIK.

Yes, the "classic" Jai example from early in development compiles a program which IIRC just prints out your score like "You got 10 points" every time you run it - but during compilation it finds the score by playing a video game, live, so if you want that program to say "You got 100 points" you need to score 100 points during compilation, once you "die" in the game your compilation finishes.

Whether this is desirable is dubious. But fundamentally it's not any more dangerous than Mara's nightly_crimes! Rust proc macro (which during compilation replaces your running compiler with a compiler that believes it is compiling the standard library and thus allows non-stable "nightly" features even though you aren't running a nightly build, then casually alters the compiler output to say that this was fine and there's nothing to worry about), or any number of other tricks which result from being powerful enough at compile time.

I doubt that D is as powerful as people would want and yet manages to ensure this can't happen. There are Rust people thinking about WASM sandboxing for this, but it's tricky.

I believe you. However, I'm not sure how opening up my build system a little more the D would open the floodgates of people evading the sandbox.

> D allows comptime dynamic allocation. So I'm confused.

with the GC, which is safe

They can, but this is a trillion dollar problem (cloud) now so it's harder than you might imagine

I think you meant to reply to me, not Walter Bright.

And yes, this is a trillion dollar problem. I may be just some bloke with an idea, trying to make it happen. However, I have the free time to try, so why not?

I meant to reply to Walter, he isn't a server guy so be probably isn't familiar with how modern systems defend against this problem.

Any Python script (or library) can run system calls.

In a language like D you could just execute the same system calls in the executable that was compiled. The assumption that you compile something and then not run it doesn't make this any safer, does it?

Pedantically, you are correct. Pragmatically, D is not getting into an arms race with people writing malicious code that can delete the users' filesystem just by compiling it. People do not expect the compiler to need to be run in a sandbox.

Can we get Jai compilers yet?

Or is it still for in-house use only?

It is in private beta. You can send an email to request access. I'm not sure of the frequency or percentage of requests that are accepted, but I have seen some people get added over time.

What's Jai?

It's a language written by Jonathan Blow, meant to replace C++ for game development.

reportedly his game "The Witness" was written in it. He has a series of video's where he explores what the language will look like as he's developing it, and has claimed he'll open source it once he feels it's ready. He wants to maintain control until he feels it's no longer half-baked.

IIRC, he demonstrates things like a music player running during compile time. It's a core tenet of his language.

> He has a series of video's where he explores what the language will look like as he's developing it

Probably the best place to see it in it's current form is his development streams on Twitch: https://www.twitch.tv/j_blow. Right now the most active streamer besides Jon is Raphael, who does a lot of work on the Jai compiler: https://www.twitch.tv/raphael_luba. Sadly, the other people I know who stream their work with it haven't been streaming much lately.

> reportedly his game "The Witness" was written in it.

No that was still written in C++. The new game that he's working on is written in Jai.

I must have gotten my wires crossed.

playing the jeopardy tune during long compilations would be awesomely hilarious (: