News for hackers. Stuff that matters.
Re:Graal and Truffle could accelerate programming language d... by lisper on Tuesday July 19, 2016 @01:36PM (#12123284)
> A way to create a new language in just a few weeks

As a Lisp programmer, I regularly create new languages in a matter of hours.

Re:Graal and Truffle could accelerate programming language d... by chrisseaton on Tuesday July 19, 2016 @01:44PM (#12123349)
Do you mean macros? Could you re-implement a language such as C as a macro? And have it be about as fast as GCC? I didn't think they were that sophisticated.
Re:Graal and Truffle could accelerate programming language d... by ScottBurson on Wednesday July 20, 2016 @02:25AM (#12127235)
A Lisp macro is simply a function from an AST (in s-expression form) to an AST. Other than that, it's an ordinary function and has full access to the power of the language.

Implementing C as a parser that builds s-expressions followed by a collection of macros to translate it into Lisp is certainly possible; I did it in 1984. I wouldn't say it was trivial, though; nor was the performance anything special (even then).

I will definitely take a closer look at Graal and Truffle.

Re:Graal and Truffle could accelerate programming language d... by pnathan on Tuesday July 19, 2016 @02:09PM (#12123576)
Yes, you could. You'd implement a program to take C code and emit an equivalent Lisp program (i.e., parse C and emit the tree into Lisp), then compile the Lisp program. The annotations and care the parser/semantic analysis code uses will provide the semantics of C.

This will, however, not "play" like C without a great deal of work. You will be dragging an entire Lisp VM around and have to shake all of that out to actually produce something as lean as a C program at the end. You're probably best suited to take the Lisp compiler and build a "independant code" generator to generate binaries that don't rely on Lisp. But I'm waving my hands, you'd need to talk to the SBCL or CCL development teams there.

Re:Graal and Truffle could accelerate programming language d... by chrisseaton on Tuesday July 19, 2016 @02:19PM (#12123655)
That doesn't sound significantly easier than what we've done to implement C using Truffle. I'm not sure I buy this argument that what we've done could have been done trivially using Lisp macros.
Re:Graal and Truffle could accelerate programming language d... by pnathan on Tuesday July 19, 2016 @02:35PM (#12123747)
Honestly, the front end would be about as hard, unquestionably! But the Lisp ecosystem is already in play, whereas you seem to have had to build that ecosystem. That is to say, it's not a major effort to scrape through the Lisp systems to understand what is going on, whereas your ecosystem with Graal etc required a great deal of work, it seems to be. Would that be correct? Or was it pretty plug and play after you had JVM bytecode?
Re:Graal and Truffle could accelerate programming language d... by qwertyuiop924 on Tuesday July 19, 2016 @04:40PM (#12124725)
Why compile to lisp in the first place? Build a nanopass framework (https://www.google.com/url?sa=t&source=web&rct=j&url=http://...) that acts on sexprs, and then emit the thing to a target of your choice. The advantage is that Lisp is really good at transforming sexprs, and macros make writing the infrastructure for that sort of compiler just that much easier.
Re:Graal and Truffle could accelerate programming language d... by pnathan on Tuesday July 19, 2016 @07:45PM (#12125793)
SBCL can compile lisp to native code. :-)
Re:Graal and Truffle could accelerate programming language d... by qwertyuiop924 on Tuesday July 19, 2016 @09:11PM (#12126190)
Well, yeah, but as you said, you have to haul the runtime around with you.
Re:Graal and Truffle could accelerate programming language d... by ScottBurson on Wednesday July 20, 2016 @02:14AM (#12127192)
It's pretty small, as runtimes go these days.
Re:Graal and Truffle could accelerate programming language d... by ktRolster on Tuesday July 19, 2016 @01:57PM (#12123492)
Could you re-implement a language such as C as a macro?

You can do that pretty quickly in IO language (admittedly, that's not Java)

Re:Graal and Truffle could accelerate programming language d... by wcrichton on Tuesday July 19, 2016 @02:02PM (#12123526)
You cannot. Because Lisp is homoiconic, you don't code in (what I personally believe to be) reasonable syntax, you code in abstract syntax trees. Lisp is so easy to metaprogram because everything is parentheses and the burden is on the user, not the parser, to determine the program's abstract syntactic structure. So, Lisp can only metaprogram its own syntax, you can't introduce a C-like syntax.
Re:Graal and Truffle could accelerate programming language d... by lisper on Tuesday July 19, 2016 @02:20PM (#12123661)
> You cannot.

You can. Reader macros let you put whatever syntax you want on top of Sexprs. For example:

    Welcome to Clozure Common Lisp Version 1.10-r16479M  (DarwinX8664)!
    ? (spark-init)
    #P"/Users/ron/devel/spark/ergolib/init.lisp"
    ? (require :parcil)
    ...
    
    ? (in-readtable infix)
    |NIL|
    ? infix(x=1.23+4.56)
    5.79
    ? sin(x*x+1)
    0.03341171
    ? 
The code for this is here: https://github.com/rongarret/ergolib
Re:Graal and Truffle could accelerate programming language d... by wcrichton on Tuesday July 19, 2016 @02:37PM (#12123764)
For sure, I was just considering normal macros.
Re:Graal and Truffle could accelerate programming language d... by lisper on Tuesday July 19, 2016 @02:52PM (#12123899)
No, you were just flat-out wrong.

> Lisp can only metaprogram its own syntax, you can't introduce a C-like syntax.

This is a flat-out false statement, and it reflects a deep but sadly common misunderstanding of how Lisp works and why it's cool. Even if Lisp did not have reader macros as a standard feature, you could still write a C compiler in Lisp more easily than you could write one in any other language. The whole concept of "metaprogramming a syntax" (so that you can talk about whether or not a language can "only metaprogram its own syntax") is non-sensical, a category error. Metaprogramming is simply writing programs that write programs. What makes Lisp cool is that it separates the syntax from the program. A Lisp program, unlike all other languages, is not text. A Lisp program is a data structure. Lisp happens to define a textual surface syntax (S-expressions) that allows you to easily convert text into the particular data structures that are Lisp programs (and also incidentally data structures that are not Lisp programs) but you can also produce these data structures in other ways, like, for example, writing programs that produce them. The textual surface syntax is a detail. Writing an infix parser in Lisp is an elementary exercise, and you could use that parser to parse C-like code whether or not you had reader macros. All reader macros let you do is seamlessly integrate that C-like syntax into the Lisp REPL rather than having to embed your new language in strings or read it from files.

Re:Graal and Truffle could accelerate programming language d... by wcrichton on Tuesday July 19, 2016 @03:10PM (#12124046)
Yes I agree with you, I am not trying to say it is impossible to write a C interpreter/compiler in Lisp. Any Turing-complete language can do that. I'm more so interested in the built-in metaprogramming capabilities, and it's evident that reader macros go far in that direction.

On a different note, would you say that Lisp's syntax is better because it makes an easy compile target (i.e. it's easier to compile C into Lisp instead of x86) or because it makes writing macros/AST transformations easier? I ask because my biggest beef with Lisp has always been that I thought its S-expressions to be less readable than other languages, but if you're arguing for Lisp's value as a compile target, then that makes a lot of sense.

Re:Graal and Truffle could accelerate programming language d... by lisper on Tuesday July 19, 2016 @03:25PM (#12124155)
> Lisp's syntax is better because it makes an easy compile target

Not quite. Lisp's syntax is (mostly) irrelevant. Lisp's design is better because you don't have to worry about syntax to use Lisp as a compile target.

There's a huge conceptual difference between

    (eval '(some code))
and (using Javascript or Python as an example)

    eval("Some code")
In JS/Python case you are passing a string to EVAL. In the Lisp case you are passing a data structure to EVAL. That data structure has been generated by the reader (by virtue of using QUOTE) but that is not the important part. You could as easily have written:

    (eval (cons (quote some) (cons (quote code) nil)))
or

    (eval (some-function-that-generates-code))
with the point being that some-function-that-generates-code does not return a string, it returns a data structure, so there is no syntax. The whole concept has evaporated because you're not dealing with strings any more. Syntax is for humans. When you have programs writing programs, syntax just gets in the way. But when your EVAL function takes a textual representation of a program as a string you have no choice but to muck with syntax. This is the reason that the misconception that syntax is essential is so widespread. It seems essential only because most programming languages don't distinguish between READ and EVAL.
Re:Graal and Truffle could accelerate programming language d... by qwertyuiop924 on Tuesday July 19, 2016 @04:29PM (#12124646)
point being, you can trivially parse code of any syntax, just like any other language. What makes lisp different is that once you have a parse tree of sexprs, you can then traverse that tree very easily: Lisp has tools built right into the language. You can then very easily transform that tree: In fact, you can write a hook whereby the code being written can provide arbitrary transformers, which are called by the compiler when seen in code: This is all a macro is: it's an API for code to hook into its own compiler: much like Rust's compiler addons. Unlike compiler addons, however, they don't depend on compiler internals: In fact, if a Lisp implementation didn't have macros and readtables, you could write a preprocessor to add them in (but gensym/hygene would be a pain to do). However, this would all be very confusing if the internal tree didn't look like the external representation. This is why Lisp is written in sexprs. The reason you see so many DSLs in lisp being written in a lispy syntax is that lisp hackers, like all other hackers, are fundamentally lazy: Since there's already a lisp syntax parser built into the language, they don't have to write their own, and macros often suffice for the most common reasons to write a DSL, which means that they didn't even have to write a compiler, they just hooked into the existing one. Besides, sexprs provide advantages even if you are writing your own language, and you can overload lisp's syntactic shortcuts for your own purposes.
Re:Graal and Truffle could accelerate programming language d... by chriswarbo on Tuesday July 19, 2016 @03:12PM (#12124065)
If you want to handle a C-like language, you just need to make a parser; once you've got an AST, that is your "Lisp syntax", so you can do all of your fancy metaprogramming, interpretation, compilation, etc.

Using a parsing framework (e.g. parser combinators, parser generators, ometa, etc.) to parse a language which has a well-specified syntax (e.g. a BNF grammar) is pretty routine and mechanical; usually it just requires a one-to-one translation of the BNF form into the parser framework's syntax, then fiddling with ordering and precedence rules until your tests pass.

Starting from scratch, I could knock out a C-like parser in maybe half an hour, e.g. using parsec or ometa. I'm sure Lisps have equivalents (I know Racket has built in support for defining new concrete synax)

Re:Graal and Truffle could accelerate programming language d... by dragonwriter on Tuesday July 19, 2016 @02:36PM (#12123761)
> Because Lisp is homoiconic, you don't code in (what I personally believe to be) reasonable syntax, you code in abstract syntax trees.

There's no reason a regular macro couldn't implement a basically C-like syntax (though it would have Lisp, not C, tokenization rules, to the extent that those are different) on what is passed as its argument. The actual call to the macro would have typical Lisp syntax, but what the macro consumes would be interpreted with whatever syntax was implemented by the macro.

(And, a reader macro could implement more deeply C-like syntax.)

Re:Graal and Truffle could accelerate programming language d... by qwertyuiop924 on Tuesday July 19, 2016 @04:33PM (#12124678)
Well, for regular macros, only if your C syntax was in a string. Otherwise, lisp would try to tokenize it, and choke. Heck, if your lisp of choice offers an eqivalent of TCL's uplevel, you wouldn't even need a macro, just a function.
Re:Graal and Truffle could accelerate programming language d... by dragonwriter on Tuesday July 19, 2016 @04:39PM (#12124713)
> Well, for regular macros, only if your C syntax was in a string. Otherwise, lisp would try to tokenize it, and choke.

Which is why I said normal (rather than reader) macros could do a "basically C-like syntax" but with Lisp tokenization rules.

Re:Graal and Truffle could accelerate programming language d... by qwertyuiop924 on Tuesday July 19, 2016 @04:42PM (#12124740)
What would that look like?
Re:Graal and Truffle could accelerate programming language d... by TJSomething on Wednesday July 20, 2016 @01:37AM (#12127104)

  ((:include "stdio.h")
  
  (const char * message = "Like this.\n")
  
  (int main ((int argc) (char * * argv))
    (printf message)
    (return 0)))
Re:Graal and Truffle could accelerate programming language d... by qwertyuiop924 on Wednesday July 20, 2016 @06:21AM (#12127947)
Ah.
Re:Graal and Truffle could accelerate programming language d... by wcrichton on Tuesday July 19, 2016 @02:39PM (#12123785)
I've never seen something like this before--do you know of any examples of this kind of macro usage?
Re:Graal and Truffle could accelerate programming language d... by dragonwriter on Tuesday July 19, 2016 @02:45PM (#12123829)
This article provides a simpler (but conceptually similar, where it comes to implementing syntax different than standard Lisp S-expression syntax) example of using reader macros to implement JSON literals in Lisp.

https://gist.github.com/chaitanyagupta/9324402

Re:Graal and Truffle could accelerate programming language d... by bostonpete on Tuesday July 19, 2016 @01:43PM (#12123337)
I'm vaguely familiar with Lisp, but I'm not a Lisp programmer. Can you explain this comment?
Re:Graal and Truffle could accelerate programming language d... by nilved on Tuesday July 19, 2016 @01:45PM (#12123363)
Lisp languages are easy to build because of their simple syntax. It's also easy to use a Lisp to build a Lisp, since it is homoiconic (this is what you're doing with the macro system.)
Re:Graal and Truffle could accelerate programming language d... by stcredzero on Tuesday July 19, 2016 @02:04PM (#12123535)
It's much the same in Smalltalk. Often, one could create a domain-specific superset of Smalltalk, complete with its own control structures.
Re:Graal and Truffle could accelerate programming language d... by pnathan on Tuesday July 19, 2016 @02:11PM (#12123591)
I was thinking that myself. If they had started with a modern Lisp implementation, they would be worlds ahead. :-/