Kinda curious what you've done "using Nim pretty intensively for several years"
I had an implementation in Python and just ported it to Nim. I have been incredibly impressed with Nim to say the least. I managed to make the Nim version orders of magnitude faster with fewer lines of code while having more features. With the excellent Nimporter library, I can easily make a Python API and have it be pip installable.
I genuinely think Nim (or similar) may be the future of my field. Combined with its JS compile target and WASM support, it lets me target my three main “platforms“: web, Python API, and high-performance CLI.
Julia was no faster than Python with my initial naive implementation. After consulting the forum and getting help, I did manage to get it running fast. My impression of Julia is that it can be incredibly fast but it takes work to get to that stage.
In contrast, with Nim, my code looked nearly identical to my Python code (i.e. quite simple) and ran fast from the start. This has been my general impression of Nim after doing a few projects in it: you write clean, readable code and get fast code out. There is no step 2.
With respect to numerical algorithms, Nim has good support for BLAS and LAPACK. I personally haven’t used them, so I can’t comment.
Nim’s metaprogramming allows for things like this [1], which is a macro library that translates between idiomatic Python and Nim. While I don’t use it myself (I find Nim’s idioms to make sense for Nim) it does make the transition a lot easier.
There is https://github.com/mratsim/Arraymancer n-dimensional tensor (ndarray) library
Native plotting library: https://github.com/Vindaar/ggplotnim
as well as many other packages.
It's an incredibly adaptable tool. I mean, just take the example of writing type safe Nim code and compiling it to JS for a web client (already handy), then writing a high performance server in the same language, sharing the same types/code, but compiled to C. We can bring together front-end and backend! :)
Another generalist ability is with libraries you can get a high cooperation with Python - you can run Nim from Python and Python from Nim - so you can merge with that ecosystem too and use existing infrastructure as required. And of course an even better situation exists with C and C++.
Personally it's been a great way to diversify my ability to reach different domains with a unified tool.
Metaprogramming is v alluring to me, but I’ve been wondering, people that write code super heavy on the metaprogramming, doesn’t revisiting code and debugging become much more taxing?
- dumpTree
- lisprRepr
- treeRepr
- toStrLit
This means that for debugging you can generate your code. Replace the macro by regular code temporarily and understand what's wrong and then fix the macros.
I don't think I have a single library without macro but I try to keep them as reasonable as you can. But sometimes you need to give a Python feel (for example to provide a Numpy interface for a multidimensional array library) or you have to do code transformation (implementing async or multithreading as a library or autovectorizing high-performance computing code or implementing a GPU or Audio DSL) and there is no way around macros.
Macros are prefered for creating dsl like https://github.com/treeform/fidget or https://github.com/pragmagic/karax
Otherwise the advice is to use them when they "pull their weight". The reason is they're a lot harder to write and even "design" them right (how they function, what code they write, etc) takes a lot of effort. And in most cases the simpler "tools" will fit your needs.
My experience debugging complex compile time transformations is pretty positive thanks to the tools available to manipulate and debug the syntax trees. For example you can `echo node.repr` to display the code it represents at any point, and going the other way you can pass code to `dumpAstGen` to display the code that will generate the AST tree and so on.
This also has the benefit of letting you quickly understand how the trees are structured by trying different bits of code, or for passing DSL syntax fragments to see the parsed tree structure for your macro to consume.
The main important thing is that stack traces at compile time get you to the place where the error is happening. How hard that is to debug depends on how complex your tree manipulation is, and how you are doing that manipulation.
For example if you're just got a load of `quote` statements that interpolate variables and you're chucking these together, the trace will usually lead to the start of the `quote` block, so if you've got a long `quote` block then you have to kind of find the error in that, but it's not too bad since the error messages are pretty precise and the language is strongly typed.
For direct AST manipulation, the error will point to the place where the error is triggered, so for example if you add a bunch of statements in a node, the error may be the place that node is added to it's parent to be 'connected' to the program as a whole. Or, if it's an attempt at a duplicate variable declaration it will point directly to the place where you're creating the `ident` that represents the variable in your macro, so that's pretty nice.
In short, yes it is more work, but it generally makes sense, is type safe, and we have tools that mean fixing stuff is often some `echo` statements to look at generated code.
Even Haskell has null pointers, if you know where to look:
https://hackage.haskell.org/package/base-4.14.0.0/docs/Forei...
You just mentioned using "nil" in a simple example which doesn't appear to be related to systems programming.
Just checked Github for some of the top Nim projects.
Forum software: https://github.com/nim-lang/nimforum/search?q=nil&unscoped_q...
Web framework: https://github.com/planety/prologue/search?q=nil&unscoped_q=...
Twitter front-end: https://github.com/zedeus/nitter/search?q=nil&type=Code
These uses of "nil" don't appear to have anything to do with systems programming.
You need nil/null when doing systems work -- and Nim is capable of that -- but I don't know the rationale for allowing 'nil' in higher level language features.
Nim interests me but this nil business seems questionable to me.
Just found this comment from the language creator,
https://github.com/nim-lang/Nim/issues/6638#issuecomment-487...
"We need the nil state to disarm pointers but that only means these can be nil and so would require a nil check before deref. Doesn't seem to be too hard nor too cumbersome."
Seems a bit dismissive to me
The new battleplan is to integrate the Z3 theorem prover into Nim as DrNim:
- https://nim-lang.org/docs/drnim.html
The first use-case will be proving array bounds at compile-time and elide them. Future use-cases include not nil tracking.
1) Echoing the other comments in this thread about uneasiness with feature creep
2) Wouldn't have been a breaking change if the language didn't have nil from the beginning
It's something you similar to MyPy for python and type annotations. The danger is if it becomes non-optional.
That said contrary too many languages, the core language doesn't need:
- threading
- async
which are huge parts to maintain, but thanks to metaprogramming you can implement them as libraries with very nice syntax as well. It also allows compiler devs to focus on what they know well, compilers and leave those 2 highly specialized areas to people who are experts (or become experts) in those.
2: There are 2 dimensions there: nil is used for C FFI, for higher-level primitives you can use Option. And C FFI was needed to bootstrap Nim with a useful set of functionality and has an escape hatch so that people could start building early on and not wait for the language. So both from a functionality and a time-to-market point of view it was necessary.
I would still recommend playing with the language for a while. It's quite practical, and the performance is very good. I've used it to write some utilities where I might otherwise use Python, but either needed more speed or needed to use it in a restricted environment where the Python interpreter wasn't available. The experience was mostly positive, and I would use it again.
It's "too good to be true" is for real. The lack of friction when developing in Nim is what makes it so fun to code in (IMO). Metaprogramming in particular is just great in Nim and give you a lot of scope for doing things that would otherwise require new language features.
> how do you cope with the ever increasing scope of the language?
I've been using Nim pretty intensively for several years. From my perspective the language design has been very stable, even since before 1.0. I don't feel like the scope is expanding, and I don't think it really needs to either because the core language is built to be expanded with metaprogramming.
None of my code breaks when updating versions (and I do a lot of heavy metaprogramming). Nothing seems to be abandoned, only refined, and there's a big effort towards backwards compatability so breaking changes are exceedingly rare. The only one I only remember was some time ago when seq (the variable sized list type à la C++ vector) became not nil, so I had to replace `if list != nil` with `if list.len > 0`, so a good change IMO.
Most of the new things have been mechanical under the hood improvements and ownership is one of those. As I understand it, when using gc refs, you'll get free extra speed, better multithreadding, and better compile-time checks. No code needs to be changed, and when it's considered ready it will replace the previous GC, and you can add extra annotations for performance. It's refinement of the GC using ownership rules rather than a whole new set of rules for your program.
The stdlib is very stable (at least from my experience), and again great effort is spent on backwards compatability. Stdlib focus is on being small and "essential", so there's a high bar for stuff getting into it, and probably why it's rare for anything to change API there.
So personally the pace of change has been great - basically my code's just got free speed boosts when it's already fast enough :)