News for hackers. Stuff that matters.
Re:The naked truth about writing a programming language (2014) by pbiggar on Saturday May 02, 2020 @05:13PM (#23054963)
This advice is quite good for certain types of programming language, if you look at the world the same was as he does. We're implementing Dark from a completely different worldview, and from that vantage point, a lot of these aren't exactly wrong, but different.

> Syntax matters

The approach we took with Dark is that there isn't a syntax, per se, in that there isn't a parser. There's certainly a textual view of Dark and so it's important that the code looks good (it currently looks only OK, in my opinion). But as a result, we have other options for minimizing keystrokes (autocomplete built-in), parsing (again, no parse), and minimizing keywords (you're allowed have a variant with the same name as a keyword, which isn't allowed in languages with parsers (well, lexers, but same idea).

> context free grammars

No parser, no need to have a grammar. His point about IDEs is great - we only support our own IDE (a controversial decision to be sure!)

> Redundancy

This is an esoteric parsing problem, that only applies if you have a parser. No parser means no syntax errors. We are left with editor errors (how does the editor have good UX) and run-time errors.

> Implementation

He's right about how hard error messages are, so I feel good about our "not parsing" approach.

> Compiler speed

Our compilation is instant. The way it's instant is:

- very small compilation units: you're editing a single function at a time, and so nothing else needs to be parsed.

- no parser: The editor directly updates the AST, so you don't have to read the whole file (there isn't a "file" concept). Even in JS, that means an update takes a few milliseconds at the most.

- extremely incremental compilation: making a change in the editor only changes the exact AST construct that's changing.

> Lowering

This is really about compilation. One thing you can do, which is what we do, is have an interpreter. Now, interpreters are slow, but we simply have a different goal with the language, which is to run HTTP requests quickly. We do run into problems with the limit of the interpreter, but we plan to add a compiler later to deal with this.

Really what I'm suggesting here is that compiled languages look at having interpreters in addition to compilers.

> i/o performance

> memory allocation

I think he's approaching this with an implicit goal of "it must be as fast as possible", which isn't necessarily a goal for all languages.

> You’ve done it, you’ve got a great prototype of the new language. Now what? Next comes the hardest part. This is where most new languages fail. You’ll be doing what every nascent rock band does — play shopping malls, high school dances, dive bars, etc., slowly building up an audience. For languages, this means preparing presentations, articles, tutorials, and books on the language. Then, going to programmer meetings, conferences, companies, anywhere they’ll have you, and show it off. You’ll get used to public speaking, and even find you enjoy it (I enjoy it a lot).

Well this is certainly correct!

Re:The naked truth about writing a programming language (2014) by tomp on Saturday May 02, 2020 @06:08PM (#23055298)
> The approach we took with Dark is that there isn't a syntax, per se, in that there isn't a parser.

Can you explain how that works? Unless Dark is a purely visual programming language (in which case I'd say that there is "syntax", it's just a bit more abstract, and in any case, it seems that visual languages haven't really caught up as an idea), I find that hard to believe.

Re:The naked truth about writing a programming language (2014) by pbiggar on Saturday May 02, 2020 @07:34PM (#23055722)
Sure. You can see it in action at http://darklang.com/launch/demo-video (also, we've gone through our waiting list, so we're pretty much adding new folks who sign up immediately, if you want to try it out).

The main idea is that you when you make a change (let's say, you type a key in the editor), that change happens directly on the AST (the internal set of objects that represent the program). So for example, if you're seeing:

    let x = 5
    x + 2

we represent that internally as `Let ("x", Int 5, BinOp ("+", Var "x", Int 2))`

In a normal programming environment, you type up text, and the parser converts it to the internal representation. With Dark, we always keep everything in the internal representation. That means, if it isn't a "syntactically" valid program, you can't create it in the first place.

In practice, this means that if you type, for example, `z` in the middle of `let`, in a traditional language you'd get an error like `lzet is not a valid token` and in Dark, the `z` just wouldn't appear (nothing would happen). So it's not possible to get a syntactically invalid program.

The overall idea with Dark is that programs are always live (and also safe). So to make that be true, we need to ensure that you don't spend a period of time in a state where we can't understand your program (such as when the syntax is invalid).

Re:The naked truth about writing a programming language (2014) by WalterBright on Sunday May 03, 2020 @05:32AM (#23057933)
> That means, if it isn't a "syntactically" valid program, you can't create it in the first place.

It's an interesting idea. I don't know if it would work for me, though. I don't write code as a stepwise refinement of a valid program, I massage the text gradually into a valid program. This is especially true when refactoring.

Re:The naked truth about writing a programming language (2014) by pbiggar on Sunday May 03, 2020 @12:09PM (#23059870)
Yeah, I think a bunch of people work like that. We've had the feedback that some people have had to change their workflow to work well with Dark.

People generally use what we call "trace-driven development", where they make a HTTP request to the code they're editing (typically, people are making APIs or backends to JS apps), and then they always have "live values" of what they're working on (similar to a REPL, or to running unit tests on a loop, but instantly).

Re:The naked truth about writing a programming language (2014) by tomp on Sunday May 03, 2020 @05:05AM (#23057849)
So, looks like it's a textual language, it clearly has a syntax. How you manage the transition from text to AST and back, that's a different story - it might be that your approach is much better! (Though on the other hand it looks similar to how CLR/Roslyn C# compiler does it.)
Re:The naked truth about writing a programming language (2014) by pbiggar on Sunday May 03, 2020 @12:05PM (#23059834)
The code always lives as AST, and the code that you see in the editor is a pretty-print of the AST. We don't go back and forth between AST and text - when you make a change in the editor, we look up the "token" where your cursor was, and use it to find the place in the AST that needs a change, and changes only that.
Re:The naked truth about writing a programming language (2014) by rgbrenner on Sunday May 03, 2020 @12:01AM (#23056778)
You have a grammar... you're just enforcing the rules when data is entered. The same as if my editor stopped accepting input if I miss typed a line until it was fixed.. or if I was required to select one of it's autocomplete suggestions.

let x = 5

"hi " ++ name

DB::set{}

Are you saying these lines from the video are not consistent throughout? Like in one part I'll type "x = 5" to assign 5 to x, but somewhere else I'll have to type "5 # x" to do the same? Like just random syntax everywhere?

If not, then you have a grammar. You should document it and treat it as such.

Re:The naked truth about writing a programming language (2014) by pbiggar on Sunday May 03, 2020 @01:39AM (#23057093)
I had to look up whether you were technically accurate (I've a PhD in compilers but parsing's not really my thing), and I think the answer is "it depends, maybe?"

To get to the root of the issue though, there is not a parser.

Well, one could consider the function that takes each keystroke to be a parser - though I don't know if that's technically correct either. So perhaps the more accurate thing to say is there is not a step that takes a textual source file and parses it.

Re:The naked truth about writing a programming language (2014) by GreedCtrl on Sunday May 03, 2020 @03:45AM (#23057539)
What would happen if you paste a complete program in all at once?
Re:The naked truth about writing a programming language (2014) by pbiggar on Sunday May 03, 2020 @12:02PM (#23059812)
That's not really how Dark works. You're not manipulating text in your editor - you use our editor, and our editor only uses ASTs. So when cutting and pasting, you're cutting and pasting AST, not text. So to answer your question, you can paste an entire handler or function at once, but it will be one you copied from another place in DArk and it will be structured.

We've looked at allowing people to paste text (eg from Stackoverflow) into Dark. We think we can do it by just inserting each character one at a time (though there's a technical flaw with that at the moment, which could be addressed). In that sense, Dark is sort-of a parser (I don't know if it technically counts).

Re:The naked truth about writing a programming language (2014) by saagarjha on Saturday May 02, 2020 @08:12PM (#23055902)
That sounds…potentially confusing? Correct me if I'm wrong, but it just continues to run the last version of the program without errors, but it won't tell you if the current version is syntactically invalid?
Re:The naked truth about writing a programming language (2014) by pbiggar on Saturday May 02, 2020 @10:54PM (#23056548)
Think I made it more confusing, not less, sorry! Did you watch the video - it's clearer after watching it.