News for hackers. Stuff that matters.
Re:Nim 1.0 by arc776 on Tuesday September 24, 2019 @06:07AM (#21058248)
I would say metaprogramming (and maybe the excellent FFI) is the huge stand-out feature for Nim.

However whilst you can compare all these languages and find a particular niche or set of features that sell them, Nim is just good at pretty much everything.

I know that sounds pretty bombastic, but you can practically pick any task and know that Nim will let you get there rapidly and performantly. That's it's ultimate strength. It's just so productive! It puts the fun back into a day job.

It's as easy to develop in as Python (great type inferrence and strong, static typing), but fast as well written C, yet (AST based) metaprogramming allows it to be higher level than even Python.

On top of low developer friction, it outputs stand-alone executables for effortless distribution. It's my first choice for scripting, for CRUD work, web stuff, game development, and embedded coding.

Doesn't hurt that it can compile to c, c++ and javascript so you can write your backend and frontend in the same language.

Also since the FFI is so great, you can effortlessly use the entire library ecosystem of C and C++.

Of course, it's not perfect, but for me it's getting pretty close.

Re:Nim 1.0 by 6gvONxR4sf7o on Tuesday September 24, 2019 @11:35AM (#21061278)
> (AST based) metaprogramming allows it to be higher level than even Python.

I'm skeptical of this claim. Python's ast tools let you do just about anything you'd want to, and just about anything you'd never want to do as well.

Re:Nim 1.0 by nunchuckninja on Tuesday September 24, 2019 @12:05PM (#21061619)
Scala and rust both compile to JS too I think.
Re:Nim 1.0 by dom96 on Tuesday September 24, 2019 @03:05PM (#21063500)
Do they have first-party JS backends though? I think that's what sets Nim apart, the Nim compiler includes the JS backend which means the JS backend is always up to date with the latest Nim developments.
Re:Nim 1.0 by nunchuckninja on Wednesday September 25, 2019 @01:41AM (#21067961)
Sorry I was wrong it's asm.js for rust. Scala does have an amazing first party backend that I love using
Re:Nim 1.0 by nickitolas on Wednesday September 25, 2019 @07:32AM (#21069673)
I didn't know there was an asm.js backend for rust. I do know there are emscripten and llvm backends for webassembly.
Re:Nim 1.0 by steveklabnik on Tuesday September 24, 2019 @01:29PM (#21062536)
Rust sorta can, but prefers to be compiled to WebAssembly.
Re:Nim 1.0 by qwerty456127 on Tuesday September 24, 2019 @12:01PM (#21061574)
This sounds like everybody should be using Nim. Why do they still use Python, C# etc and why is Nim still so rare then?
Re:Nim 1.0 by dixego on Tuesday September 24, 2019 @01:58PM (#21062895)
Having had it's 1.0 release a day ago might have something to do with it.
Re:Nim 1.0 by greyfade on Tuesday September 24, 2019 @05:02PM (#21064590)
Python has been around since, what, 1994? C# goes back to '99 or '00 or something like that.

Nim goes back to maybe 2008 and only hit 1.0 today.

Re:Nim 1.0 by marble-drink on Tuesday September 24, 2019 @07:38AM (#21058846)
Talk of metaprogramming intrigues me. I'd like to hear what a Lisp user makes of it because I find non-Lisp users are usually amazed by any metaprogramming at all and can't be as critical about it.
Re:Nim 1.0 by PMunch on Tuesday September 24, 2019 @09:19AM (#21059805)
Not a Lisp user, but a heavy Nim macro user. Nim macros are AST based, which means that after the parser part of the compiler have read the code and created a tree structure of it your macro is called on this tree. Then you can modify this tree to your hearts desire and return a new tree that the Nim compiler will splice back into the main tree before continuing. This means that you rewrite pretty much anything into pretty much anything else. You also have access to reading files on the machine, or doing pretty much anything else that Nim offers. For examples of what you can do with it you can have a look at my macro to create GUIs: https://peterme.net/cross-platform-guis-and-nim-macros.html, my article on how I use macros to create more read- and maintainable code: https://peterme.net/metaprogramming-and-read-and-maintainabi... or perhaps see how Protobuf was written as a string parsing macro which reads a Protobuf specification and spits out all the types and procedures you need to use it: https://github.com/PMunch/protobuf-nim
Re:Nim 1.0 by lispm on Wednesday September 25, 2019 @06:34AM (#21069390)
> which means that after the parser part of the compiler

Lisp works slightly different. The macro form is an expression (a list) with the macro operator as the first element. The next elements in the expression can be anything as data.

Re:Nim 1.0 by thsealienbstrds on Wednesday September 25, 2019 @05:56AM (#21069239)
Disclaimer: I only did a hobby project in Lisp once. But I did use some of the macro functionality.

Judging by some of the comments here, it seems like the macro system has a similar approach as Lisp's macro system, which is also AST-based. Something I don't see here is macros that generate other macros, but the question is how much you really want that anyway (when I did that, I thought the syntax was horribly complicated because of all the quoting). I know Lisp also has reader macros (they run before the parser) that allow you to effectively change the language syntax, but I didn't use those.

Re:Nim 1.0 by arc776 on Wednesday September 25, 2019 @07:18AM (#21069609)
> Something I don't see here is macros that generate other macros, but the question is how much you really want that anyway

You can do that in Nim in a readable way.

  import macros
  macro genMacro(name: untyped): untyped =
    result = quote do:
      macro `name`: untyped =
        result = quote do:
          echo "Foo"

  genMacro(bar)
  bar # Generate and perform the echo
Surprisingly I've actually used this kind of thing!

In one of my projects I use a macro to parse a set of types for fields and generate constructor macros for them.

The generated constructor macro passes through the parameters it's given to the default built-in constructor but does some setup before.

The final generated code is a normal built-in construction without proc calling yet with special fields initialised automatically.

Re:Nim 1.0 by thsealienbstrds on Wednesday September 25, 2019 @09:03AM (#21070332)
> In one of my projects I use a macro to parse a set of types for fields and generate constructor macros for them.

Funny, I did a similar thing! :)

Re:Nim 1.0 by lispm on Wednesday September 25, 2019 @06:25AM (#21069358)
Lisp macros are not based on an AST. The macro forms in Lisp need to be valid s-expressions and begin with a macro operator. That's it.
Re:Nim 1.0 by thsealienbstrds on Wednesday September 25, 2019 @11:37AM (#21071779)
Still, isn't it AST-based in the sense that the input of a Lisp macro is effectively a parsed syntax tree, just as in Nim?
Re:Nim 1.0 by lispm on Wednesday September 25, 2019 @12:18PM (#21072183)
The only restriction is that it is a s-expression: a possibly nested list of data. There is no particular programming language syntax it is parsed against - it's parsed as data.

For example this is a valid Lisp macro form

    (loop for i below 10 and j downfrom 20
          when (= (+ i j) 9) sum i into isum of-type integer
          finally (return (+ j isum)))

    It returns 10.
The LOOP macro parses it on its own - it just sees a list of data. Lisp has other than that no idea what the tokens mean and what syntax the LOOP macro accepts.

    CL-USER 142 > (defmacro my-macro (&rest stuff)
                     t)
    MY-MACRO

    CL-USER 143 > (my-macro we are going to Europe and have a good time)
    T
Works.
Re:Nim 1.0 by kazinator on Wednesday September 25, 2019 @12:27PM (#21072306)
The data has an AST. The AST is made of conses, and atoms. The atoms have a large variety of types. They are not "tokens", but objects. Tokens only exist briefly inside the Lisp reader. 123 and |123| are tokens; one converts to an integer node, one to a symbol node. (1 (2 3)) has an AST which is

    CONS
   /      \
  FIXNUM   CONS
  |        /   \
  1      CONS   SYMBOL
        /     \     \ 
      FIXNUM   CONS  NIL
        |      /   \
        2   FIXNUM  SYMBOL
              |        |
              3        NIL
Re:Nim 1.0 by lispm on Wednesday September 25, 2019 @02:41PM (#21073824)
> The data has an AST.

(not (eq 'has 'is))

But for an Abstract Syntax Tree for code we have more categories: function, operator, call, control structure, variable, class, ...

Re:Nim 1.0 by kazinator on Wednesday September 25, 2019 @07:22PM (#21076658)
That kind of tree is stipulated a particular form of compiler (well, parser) writing dogma revolving around C++/Java style OOP. You set up classes with inheritance representing various node types; everything has methods, and "visitors" walk the tree, and various nonsense like that.

If our code walker dispatches on pattern matches on the nested list structure of conses and atoms, we don't need that sort of encapsulated data structuring. The shapes of the patterns are de facto the higher level AST nodes. The code walking pattern case that recognizes (if test [then [else]]) is in fact working with an "if node". That AST node isn't defined in a rigid structure in the underlying data, but it's defined in the pattern that is applied to it which imposes a schema on the data.

If that's not an AST node, that's like saying that (1 95000) isn't an employee record; only #S(employee id 1 salary 95000) is an employee record because it has a proper type with a name, and named fields, whereas (1 95000) "could be anything".

Re:Nim 1.0 by lispm on Thursday September 26, 2019 @01:32AM (#21078483)
Syntax trees are no nonsense and go way back before Java and visitors were hip.

The code walker is just another parser. It needs to know the Lisp syntax. It needs to know which parts of a LET form is a binding list, what a binding list looks like, it needs to know where declarations are and where the code body ist. It can then recognize variables, declarations, calls, literal data, etc, It needs to know the scope of the variables etc. Nothing of that is encoded in the LET form (since it is no AST), and needs to be determined by the code walker. Actually that's one of the most important uses: finding out what things are in the source code. Lisp does not offer us that information. That's why we need an additional tool. A code walker may or may not construct an AST.

No, (1 950000) is not an employee record. Only your interpretation makes it one. Other than that our default Lisp interpretation based on s-expressions: it's a list of two numbers. In terms of the machine it's cons cells, numbers, nil. Without further context, it has no further meaning.

Re:Nim 1.0 by kazinator on Thursday September 26, 2019 @01:45AM (#21078546)
Note that deciphering type tag info in a Lisp value can be called "parsing". Cases occur: for some values we have to chase a pointer into a heap to get more type info.

A Lisp function call does "parsing". (foo 1 2 3) has to figure out dynamically whether a (lambda (&rest args)) is being called or (lambda (a b c)) or (lambda (a b &optional c (d 42)) or whatever.

The #S(employee id 1 salary 95000) object also isn't an employee record without context and interpretation.

> it needs to know where declarations are and where the code body ist

The syntax can be subject to a fairly trivial canonicalizing pass, after which all these things are at fixed positions:

  (let ((a 3) b) (foo a)) ---canon-->  (let ((a 3) (b)) (declare) (foo a))
Now the variables are all pairs to which we can blindly apply car and cadr, the declarations are at caddr and the body forms at cdddr.
Re:Nim 1.0 by lispm on Thursday September 26, 2019 @03:10AM (#21078909)
> Note that deciphering type tag info in a Lisp value can be called "parsing".

No. You are still operating on the level of s-expressions, a data format. The type-tag of LET is SYMBOL.

Here we have some Lisp code in the form of an s-expression:

   (let ((let 'let))
     ((lambda (let)
        (let ((let let))
          let))
      let))
All above LET have the same type tag, but in terms of syntax they have a different purpose in the form above: we have special operators, variable declarations, variable usage, data objects. I can't just car/cdr down the lists and call TYPE-OF. This always returns SYMBOL for LET.

On the level of a syntax tree we would want to know what it is in terms of syntactic categories: variable, operator, data object, function, macro, etc. Lisp source code has no representation for that and we need to determine that by parsing the code.

Re:Nim 1.0 by kazinator on Thursday September 26, 2019 @12:03PM (#21082312)
The type tag is to be "parsed out" in some implementation specific way. For instance, we first look at the LSB of the Lisp value. If we find a 0 there, then it's a pointer; we dereference the pointer into some heap record, where we retrieve a second-level type tag. Except that, perhaps, if the whole thing is 0 bits, then it's nil; we must not chase that as a pointer. And we find a 1 in in the LSB, then perhaps we look at other surrounding bits for the type. All of this examination of bits with irregularities and cases looks like parsing.
Re:Nim 1.0 by thsealienbstrds on Wednesday September 25, 2019 @04:31PM (#21075043)
Fair enough I guess. If the Lisp macro system only sees parsed s-exps, in principle you still need to figure out for yourself what kind of construct you are dealing with on a higher level of abstraction. Nim's macro system seems to be operating on a higher level of abstraction in that regard.
Re:Nim 1.0 by arc776 on Tuesday September 24, 2019 @11:48AM (#21061427)
Metaprogramming is one of the core ideals in the birth of the language so is well supported. Personally I've not used Lisp, so can't comment there, but it is on the list of influences on the homepage.

Essentially there's a VM that runs almost all the language barring importc type stuff, and you can chuck around AST node objects to create code, so metaprogramming is done in the core Nim language. You can read files so it's easy to slurp files and use them to generate code or other data processing at compile-time.

Several simple utility operations in the stdlib make things really fluid; the easy ability to `quote` blocks of code to AST, and outputting nodes and code to string. This lets you both hack something together quickly and learn over time how the syntax trees work.

Quoting looks like this:

  macro repeat(count: int, code: untyped): untyped =
    quote do:
      for i in 0..<`count`:
        `code`

  repeat(10):
    echo "Hello"
Inspecting something's AST can be done with dumpTree:

  dumpTree:
    let
      x = 1
      y = 2
    echo "Hello ", x + y
To save even more effort, there's also dumpAstGen which outputs the code to create that node manually, and even a dumpLisp!

You can display ASTs inside macros:

  macro showMe(input: untyped): untyped =
    echo "Input as written:", input.repr
    echo "Input as AST tree:", input.treerepr
    result = quote: `input` + `input`
    echo result.repr
    echo result.treerepr
So it's really easy to debug what went wrong if you're generating lots of code.

Since untyped parameters to a macro don't have to be valid Nim code (though they still follow syntax rules) you can make your own DSLs really easily and reliably because any input is pre-parsed into a nice tree for you.

Here's a contrived example of some simple DSL that lets you call procs and store their results for later output:

  import macros, tables
  
  macro process(items: untyped): untyped =
    result = newStmtList()
    # Create hash table of 'perform' names to store their result variables.
    var performers: Table[string, NimNode]
  
    for item in items:
      let
        command = item[0]
        param = item[1]
        paramStr = $param
  
      case $command
      of "perform":
        # Check if we've already generated a var for holding the return value.
        var node = performers.getOrDefault(paramStr)
        if node == nil:
          # Generate a variable name to store the performer result in.
          # genSym guarantees a unique name.
          node = genSym(nskVar, paramStr)
          performers.add(paramStr, node)

          # Add the variable declaration
          result.add(quote do:
            var `node` = `param`()
          )
        else:
          # A repeat performance, we don't need to declare the variable and can overwrite the
          # value in the fetched variable.
          result.add(quote do:
            `node` = `param`()
            )
      of "output":
        let node = performers.getOrDefault(paramStr)
        if node == nil: quit "Cannot find performer " & paramStr
        result.add(quote do:
          echo `node`)
      else: discard
    # Display the resultant code.
    echo result.repr

  proc foo: string = "foo!"
  proc bar: string = "bar!"

  process:
    perform foo
    perform bar
    perform foo
    output foo
    output bar
The generated output from process looks like:

  var foo262819 = foo()
  var bar262821 = bar()
  foo262819 = foo()
  echo foo262819
  echo bar262821
Re:Nim 1.0 by arc776 on Tuesday September 24, 2019 @04:27PM (#21064274)
In hindsight maybe the "performers" table would be better named "resultVars", as that's what it really represents.
Re:Nim 1.0 by pjmlp on Wednesday September 25, 2019 @05:34AM (#21069143)
We do metaprogramming all the time on JVM and .NET languages.