However whilst you can compare all these languages and find a particular niche or set of features that sell them, Nim is just good at pretty much everything.
I know that sounds pretty bombastic, but you can practically pick any task and know that Nim will let you get there rapidly and performantly. That's it's ultimate strength. It's just so productive! It puts the fun back into a day job.
It's as easy to develop in as Python (great type inferrence and strong, static typing), but fast as well written C, yet (AST based) metaprogramming allows it to be higher level than even Python.
On top of low developer friction, it outputs stand-alone executables for effortless distribution. It's my first choice for scripting, for CRUD work, web stuff, game development, and embedded coding.
Doesn't hurt that it can compile to c, c++ and javascript so you can write your backend and frontend in the same language.
Also since the FFI is so great, you can effortlessly use the entire library ecosystem of C and C++.
Of course, it's not perfect, but for me it's getting pretty close.
I'm skeptical of this claim. Python's ast tools let you do just about anything you'd want to, and just about anything you'd never want to do as well.
Nim goes back to maybe 2008 and only hit 1.0 today.
Lisp works slightly different. The macro form is an expression (a list) with the macro operator as the first element. The next elements in the expression can be anything as data.
Judging by some of the comments here, it seems like the macro system has a similar approach as Lisp's macro system, which is also AST-based. Something I don't see here is macros that generate other macros, but the question is how much you really want that anyway (when I did that, I thought the syntax was horribly complicated because of all the quoting). I know Lisp also has reader macros (they run before the parser) that allow you to effectively change the language syntax, but I didn't use those.
You can do that in Nim in a readable way.
import macros
macro genMacro(name: untyped): untyped =
result = quote do:
macro `name`: untyped =
result = quote do:
echo "Foo"
genMacro(bar)
bar # Generate and perform the echo
Surprisingly I've actually used this kind of thing!In one of my projects I use a macro to parse a set of types for fields and generate constructor macros for them.
The generated constructor macro passes through the parameters it's given to the default built-in constructor but does some setup before.
The final generated code is a normal built-in construction without proc calling yet with special fields initialised automatically.
Funny, I did a similar thing! :)
For example this is a valid Lisp macro form
(loop for i below 10 and j downfrom 20
when (= (+ i j) 9) sum i into isum of-type integer
finally (return (+ j isum)))
It returns 10.
The LOOP macro parses it on its own - it just sees a list of data. Lisp has other than that no idea what the tokens mean and what syntax the LOOP macro accepts. CL-USER 142 > (defmacro my-macro (&rest stuff)
t)
MY-MACRO
CL-USER 143 > (my-macro we are going to Europe and have a good time)
T
Works. CONS
/ \
FIXNUM CONS
| / \
1 CONS SYMBOL
/ \ \
FIXNUM CONS NIL
| / \
2 FIXNUM SYMBOL
| |
3 NIL(not (eq 'has 'is))
But for an Abstract Syntax Tree for code we have more categories: function, operator, call, control structure, variable, class, ...
If our code walker dispatches on pattern matches on the nested list structure of conses and atoms, we don't need that sort of encapsulated data structuring. The shapes of the patterns are de facto the higher level AST nodes. The code walking pattern case that recognizes (if test [then [else]]) is in fact working with an "if node". That AST node isn't defined in a rigid structure in the underlying data, but it's defined in the pattern that is applied to it which imposes a schema on the data.
If that's not an AST node, that's like saying that (1 95000) isn't an employee record; only #S(employee id 1 salary 95000) is an employee record because it has a proper type with a name, and named fields, whereas (1 95000) "could be anything".
The code walker is just another parser. It needs to know the Lisp syntax. It needs to know which parts of a LET form is a binding list, what a binding list looks like, it needs to know where declarations are and where the code body ist. It can then recognize variables, declarations, calls, literal data, etc, It needs to know the scope of the variables etc. Nothing of that is encoded in the LET form (since it is no AST), and needs to be determined by the code walker. Actually that's one of the most important uses: finding out what things are in the source code. Lisp does not offer us that information. That's why we need an additional tool. A code walker may or may not construct an AST.
No, (1 950000) is not an employee record. Only your interpretation makes it one. Other than that our default Lisp interpretation based on s-expressions: it's a list of two numbers. In terms of the machine it's cons cells, numbers, nil. Without further context, it has no further meaning.
A Lisp function call does "parsing". (foo 1 2 3) has to figure out dynamically whether a (lambda (&rest args)) is being called or (lambda (a b c)) or (lambda (a b &optional c (d 42)) or whatever.
The #S(employee id 1 salary 95000) object also isn't an employee record without context and interpretation.
> it needs to know where declarations are and where the code body ist
The syntax can be subject to a fairly trivial canonicalizing pass, after which all these things are at fixed positions:
(let ((a 3) b) (foo a)) ---canon--> (let ((a 3) (b)) (declare) (foo a))
Now the variables are all pairs to which we can blindly apply car and cadr, the declarations are at caddr and the body forms at cdddr.No. You are still operating on the level of s-expressions, a data format. The type-tag of LET is SYMBOL.
Here we have some Lisp code in the form of an s-expression:
(let ((let 'let))
((lambda (let)
(let ((let let))
let))
let))
All above LET have the same type tag, but in terms of syntax they have a different purpose in the form above: we have special operators, variable declarations, variable usage, data objects. I can't just car/cdr down the lists and call TYPE-OF. This always returns SYMBOL for LET.On the level of a syntax tree we would want to know what it is in terms of syntactic categories: variable, operator, data object, function, macro, etc. Lisp source code has no representation for that and we need to determine that by parsing the code.
Essentially there's a VM that runs almost all the language barring importc type stuff, and you can chuck around AST node objects to create code, so metaprogramming is done in the core Nim language. You can read files so it's easy to slurp files and use them to generate code or other data processing at compile-time.
Several simple utility operations in the stdlib make things really fluid; the easy ability to `quote` blocks of code to AST, and outputting nodes and code to string. This lets you both hack something together quickly and learn over time how the syntax trees work.
Quoting looks like this:
macro repeat(count: int, code: untyped): untyped =
quote do:
for i in 0..<`count`:
`code`
repeat(10):
echo "Hello"
Inspecting something's AST can be done with dumpTree: dumpTree:
let
x = 1
y = 2
echo "Hello ", x + y
To save even more effort, there's also dumpAstGen which outputs the code to create that node manually, and even a dumpLisp!You can display ASTs inside macros:
macro showMe(input: untyped): untyped =
echo "Input as written:", input.repr
echo "Input as AST tree:", input.treerepr
result = quote: `input` + `input`
echo result.repr
echo result.treerepr
So it's really easy to debug what went wrong if you're generating lots of code.Since untyped parameters to a macro don't have to be valid Nim code (though they still follow syntax rules) you can make your own DSLs really easily and reliably because any input is pre-parsed into a nice tree for you.
Here's a contrived example of some simple DSL that lets you call procs and store their results for later output:
import macros, tables
macro process(items: untyped): untyped =
result = newStmtList()
# Create hash table of 'perform' names to store their result variables.
var performers: Table[string, NimNode]
for item in items:
let
command = item[0]
param = item[1]
paramStr = $param
case $command
of "perform":
# Check if we've already generated a var for holding the return value.
var node = performers.getOrDefault(paramStr)
if node == nil:
# Generate a variable name to store the performer result in.
# genSym guarantees a unique name.
node = genSym(nskVar, paramStr)
performers.add(paramStr, node)
# Add the variable declaration
result.add(quote do:
var `node` = `param`()
)
else:
# A repeat performance, we don't need to declare the variable and can overwrite the
# value in the fetched variable.
result.add(quote do:
`node` = `param`()
)
of "output":
let node = performers.getOrDefault(paramStr)
if node == nil: quit "Cannot find performer " & paramStr
result.add(quote do:
echo `node`)
else: discard
# Display the resultant code.
echo result.repr
proc foo: string = "foo!"
proc bar: string = "bar!"
process:
perform foo
perform bar
perform foo
output foo
output bar
The generated output from process looks like: var foo262819 = foo()
var bar262821 = bar()
foo262819 = foo()
echo foo262819
echo bar262821From Wikipedia article of each language:
Rust: First appeared July 7, 2010; 9 years ago
Go: First appeared November 10, 2009; 9 years ago
Nim: First appeared 2008; 11 years ago type
# A `ref` type is GC and will use the heap.
MyGCType = ref object
fieldA: int
# Otherwise ALL types are stack based.
MyStackType = object
fieldA: int
Also GC is deferred, so if you use a GC type in a local scope that doesn't escape, you don't pay for reference counting.The only other type that use GC is `seq` (equivilent to C++ vectors) IIRC.
The standard library uses seqs in various places so if you fully turn the GC off using the compiler switch --gc:none you'll get warnings for things you use that will leak. There's no GC 'runtime' stuff that you need though.
However in my experience all you need to do is just not use refs, and make your own ptr based seq (there's probably a library for this but its trivial to implement).
Nim's GC is thread-local (so no stop-the-world issues), only triggered on allocate, and has realtime support via enforcing collection periods. Plus you can use other GCs if you wish (eg Boehm).
More info about the GC here: https://nim-lang.org/docs/gc.html
So similar to C# (2000)? A useful feature to be sure, but not a major innovation.
> The standard library uses seqs in various places so if you fully turn the GC off using the compiler switch --gc:none you'll get warnings for things you use that will leak. There's no GC 'runtime' stuff that you need though.
Running with the GC off (and accepting the leaks) was already a standard managed-language technique though.
> Nim's GC is thread-local (so no stop-the-world issues)
Well no wonder it has nice properties if it avoids all of the hard problems! What happens when you pass references between threads?
> only triggered on allocate, and has realtime support via enforcing collection periods.
Your link describes the realtime support as best-effort, and implies that it doesn't work for cycle collection. So honestly this doesn't seem to offer much over e.g. the tuneable GC of the JVM (which admittedly made a massive mistake in choosing defaults that prioritised batch throughput rather than latency).
I do appreciate the information, and hope this isn't coming off as overly confrontational. But honestly it sounds like Nim is overselling things that are mostly within the capabilities of existing managed languages (maybe even behind them if the "GC" is only thread-local and/or not cycle-collecting).
The language does have pointers, and will let you do any C-style memory unsafe whatever you want to do with them. However, it doesn't have any library calls that I'm aware of that are equivalent to C's malloc() and free(). You'd have to supply your own.
There are also ambitions, of introducing real non-GC memory-safe memory management. Probably something along the lines of how Rust does it. Those haven't come to fruition yet, though.
So, long story short, yes you can completely disable GC, but I think that its capabilities on that front are somewhat overstated.
The malloc/free equivilents are in the system module: https://nim-lang.org/docs/system.html#alloc%2CNatural
GC types are copied over channels with threads, or you can use the usual synchronisation primitives and pass pointers.
As you say, thread-locality avoids the hard problems and this is a good default - I would argue that most of the time you want your data being processed within a thread and the communication between them to be a special case.
Certainly, there's a lot of talk of adding some sugar to threading, and Nim does offer some interesting tastes, such as the parallel statement: https://nim-lang.org/docs/manual_experimental.html#parallel-...
The performance of the default GC is good to very good, the JVM is almost certainly better in most cases, however this is comparing apples to oranges; it's a different language.
Nim's GC pressure is much lower in most cases, not least because everything defaults to stack types which not only don't use GC but tend to be much better for cache coherence due to locality. Using ref types is not required unless you use inheritance, however the language does encourage composition over inheritance, despite providing full OO capabilities, so you find inheritance isn't as needed as in other languages.
Plus it's not really much different to using refs to drop down to pointer level and avoid the GC without disabling it:
type
DataObj = object
foo: int
Data = ptr DataObj
proc newData: Data = cast[Data](alloc0(DataObj.sizeOf))
proc free(data: Data) = data.deAlloc
var someData = newData()
someData.foo = 17
echo someData.repr
# the echo outputs eg: ptr 000000000018F048 --> [foo = 17]
someData.free
All this means that you can 'not use the GC' whilst not disabling it. I am a performance tuning freak and I still use GC seqs all the time because the performance hit is actually using the heap instead of the stack, and worse, the actual allocation of heap memory - regardless of the language. The GC overhead is miniscule even with millions of seqs and would only even come into play when allocating memory inside loops. At that point, it's not the GC that's an issue, but the allocation pattern.Again though, it's nice to be able to drop down to pointers easily when you do need every clock cycle.
Languages like Nim allow for having the productivity of relying on GC's help, with language features for performance freaks available, when they really need to make use of them.
Java not offering a Modula-3 like feature set has tainted a whole generation to think that GC == Java GC.
EDIT: grammar errors
We can extend this pattern:
> has tainted a whole generation to think that OOP == Java OOP
Or do you mean something else?
In what concerns C#, besides GC, you get access to off heap unamaged allocations, low level byte manipulations, value types, inlined vector allocations, stack allocation, struct aligments, spans.
All GCs have eventually to stop the world, but they aren't all made alike, and it is up to developers to actually take use of the language features for writing GC-free code.
Not entirely true. Erlang's BEAM definitely doesn't need to (unless you define "world" to be a single lightweight process). Perl6's MoarVM apparently doesn't need to, either.
Doing it in some local context is a way to minimize overall process impact.
Just like reference counting as GC algorithm does introduce pauses, specially if implemented in a naive way. More advanced ones end up being a mini tracing GC algorithm.
Regardless, having any form of automatic memory management in system languages is something that should have already become standard by now.
Erlang doesn't require stopping the world because every Erlang process is isolated from the others (no shared state at all, let alone mutable shared state). I don't know off-hand how Perl avoids it.
Not at all. Nim is an example.
That's a big advantage over Rust, D, OCaml, and similar languages without this feature.
Ultimately if you're really interested in performance, regardless of the stages of compilation, the juice is the machine code output at the end.
In terms of guarantees, those should be satisfied higher up in the language itself, with C generation being output according to Nim's CGen spec. The compiler is fully open source though and easy to dig into.
Having said that, the CGen output is fairly readable if you're familiar with C and I must say I've investigated it when I wasn't sure how something was generated.
Those languages tend to be a lot simpler than C, both in terms of what constructs they offer and in terms of how simply they translate into (platform-specific) assembly language. I'm not against the idea of intermediate languages in general, but IME C is the worst of both worlds: more complicated than most high-level languages and most assembly languages.
> In terms of guarantees, those should be satisfied higher up in the language itself, with C generation being output according to Nim's CGen spec. The compiler is fully open source though and easy to dig into.
The problem is being confident that those guarantees are preserved all the way down. It's very hard to be confident of the properties that any given piece of C code has, because it's extremely rare for C code to be 100% standards compliant and different compilers do radically different things with the same code. And it's hard to reason about the effects of changes because C compilers are so complicated: maybe you understand the Nim compiler and can see how two similar Nim functions will generate similar C code, but that doesn't help you much when two similar C functions can be compiled to radically different assembly (which happens a lot).
Indeed several languages compile to C; I regard that as a downside for all those languages. I'm not sure what your point is?
If you care about the assembly a piece of code generates, you can just look at that.
The generated C code that Nim emits looks like what it is: boring automatically-generated C code.
If you wanted to debug the compiler, you could take a look at that, but most users don't have to.
To you, that is
I'm lucky to have folks here to set me straight. :)
Nim is nowhere close to that. Most people describe it as a very fast statically typed Python.
https://trends.google.com/trends/explore?geo=US&q=%2Fm%2F09g...
I'm only comparing it to those languages because of the direct comparisons many others have many in this thread. And popularity != quality, but age is also a very poor measure of success.
I am not a contributor, but I also quickly fell in love with the language. I felt incredibly productive and being able to build tiny binaries that were cross platform was awesome compared to my old Golang binaries which tend to be heavy (bundling everything into a single binary). Also coming from a web background with little system/application programming knowledge I was able to pick it up really quick. It really is a "better python" in my opinion. The language is incredibly easy to read and write, has really good JSON support (which is a big plus for me) and is incredibly fast. To add more reasons to try, Visual Studio Code has a really good plugin for the language.
Is it corporately affiliated? - No, it was created by Andreas, and has stayed independent. But it has corporate backing, which helps pay for development, and they do get a certain prioritisation in what gets implemented. But no closed door stuff.
What languages is it similar to? - Depends on what kind of similarities. Syntactically it will remind you of Python or maybe even Ruby. But when you use it the powerful type system might remind you of Ada, the macro system possibly of Lisp, the run-time speed of C/C++ or Rust, and the compilation time of, well, not C++ (it's really fast).
What is the motivation of its creators? - As far as I know the initial motivation was to have something that was as usable as Python and languages like that, but which was actually fast and type-safe. For me as a contributor these features allow me to focus on higher level things so I tend to mostly use the macro and type system to create wonderful abstractions that allow me to create nice readable code quickly.
What is the philosophy driving its design? - Haven't seen any guys with long white beards, pipes, and stacks of book in the community, so I'm not entirely sure there is an official philosophy. The design is mostly based on giving tools to the developer, and making sure they do "the right thing". If the developer then uses these tools to chop their arm of then that is their prerogative, but generally not advised.
Is it intended to be an eventual competitor or replacement for languages XYZ? - Well many people come to Nim from Python, seeking better performance, or from C or JS seeking better usability. While Nim doesn't try to position itself as a direct replacement or competitor for any one language in particular it is a probably candidate for most if not all of them.
Where does it sit on the spectrum between raw assembly and MS Access? - In terms of abstraction somewhere high up above with Lisp and friends (no personal experience with MS Access, so not sure where that actually ranks). The core language is pretty straight forward, but with the macro system the sky is the limit!
Is it intended to be a "systems language" eg can you write a kernel or device driver using it? - Yes, Nim does call itself a systems programming language, and there have been toy kernels written in it (not sure about drivers though, but I wouldn't be surprised). It can also run on micro-controllers and other embedded systems. Pretty much as long as you can run C, you can probably run Nim. That being said it also compiles down to JS, and has a lot of features that allow it to be used for anything from web-sites, through games and applications, to micro-controllers and kernels.
Does it require a VM? - Trick question, but yes and no. The compiler implements a VM to run Nim code on compile-time for the macro system, but this VM is not present at run-time.
Is there corporate support available? - It is possible to get a contract with the core developers to get support yes. Not that I've ever felt the need for any further support than the excellent IRC channel (which is also bridged to Gitter and Discord).
How established is the developer community? - The aforementioned IRC channel is very active, as well of course as the GitHub repository. There is also a yearly meetup at FOSDEM, and talks, articles, and videos about the language and development is coming out all the time.
Are there books and courses? - There is so far only one physical book, Nim in Action, and it's really good! As far as courses go there is no active course with a teacher and course material (as far as I know). But there are tutorials for all levels of programming skills, from never programmed before to picking it up as your n-th language. There are also more interactive things like Excercism where you can solve problems and have people from the community look over your solutions. Other than that people will often happily look over and comment on your project if you post in on Reddit or drop a link in the IRC channel.
Does it have garbage collection? - Yes, by default Nim has a garbage collector, but it is tuneable so you can control when and for how long it runs. And you can disable it completely and use manual memory allocation, for when you're doing kernels and micro-controller work for example. The garbage collector is also per-thread, meaning that running multi-threaded will not stop your entire application when it's collecting garbage. You also have an option regarding which garbage collector you want to use, all with different performance considerations, but in general all really fast.
Is it a compiled language? - Yes. Nim itself compiles down to C/C++/Obj-C or even JS. Then it is passed on to the target compiler to create native binaries, or in the case of JS left to either be run by node or loaded into a web-page.
Is it a strongly typed language? - Yes. Nims type system is strong and flexible. The basics similar to Adas type system. It also has type-inference however, so you might not guess that it is strongly typed by looking at the examples.
What are the three perfect use cases? - Hmm, this one is tough. I use Nim for pretty much everything, from scripting simple things on my own machine, to creating and deploying web-sites, graphical applications and games. But judging from the community I think the top three use cases are: High-performance computations, Web-dev, and games. The primary financial backer of Nim, Status, also uses it to create an Ethereum client focused on mobile devices. But not sure which category to bunch that into.
How stable is the API? - Now with the release of v1 the API of the core language and stdlib will be stable moving forward on the v1 branch. This means bug fixes will be back-ported, and any new features will only be added if they don't break this promise.
What's the package management story? - Nim has its own package manager, Nimble, and over 1000 packages ready to go with everything from bindings to popular C libraries to large pure Nim libraries and applications.
Does it have a reasonable standard library? - Indeed! The standard library is large, and has pretty much everything you need.
Is there editor support for color coding and linting? - Yes, there are plug-ins for various editors (most use the VSCode one, I personally prefer Vim), and also an LSP implementation if you want these features in an editor that doesn't yet support them.
What's the job market look like? - Not huge to be perfectly honest, there are a couple of opportunities around. But I was surprised to find people at my current job using it for some tinkering and internal tools. I'm personally lucky enough that I have fairly large autonomy in how I do my work, and I'm able to use Nim.
Is the documentation well-written? - Personally I find them pretty good, and recently they have been read through and improved, so they are better than ever. Nim also offers document comments directly in the code, so many libraries you find will be nice and documented as well.
And most importantly, if we all got in the same room, would I want to hang out with the people excited about this language? - I've tried this at FOSDEM for two years now and they are a wonderful bunch!
I think that the [nim github tagline](https://github.com/nim-lang/Nim) expresses well the philosphy: "Nim is a compiled, garbage-collected systems programming language with a design that focuses on efficiency, expressiveness, and elegance (in that order of priority)."
also Araq's blog post on 1.0 explains well how the guiding principles remained the same over time:
"When I started Nim's development I had a small simple language in mind that compiles to C; its implementation should not be more than 20,000 lines of code. The core guideline has always been that Nim should be a small language with a macro system, which should be capable of extending Nim with all the features that the small core is missing.
The current compiler plus the parts of the standard library it uses has roughly 140,000 lines of code, runs on a plethora of operating systems and CPU architectures, can also compile Nim code to C++ and JavaScript, and Nim's meta programming capabilities are top of the class. While the language is not nearly as small as I would like it to be, it turned out that meta programming cannot replace all the building blocks that a modern language needs to have.
For example, while Nim manages to implement async via its macro system, the macro system needs to be able to compile code into a state machine. And these state machines need some form of goto and a way to capture their environment. So Nim's core needed to grow so called "closure iterators" to enable this.
Furthermore, we don't really know yet how to leverage a macro system in order to give us extensibility on the type system level, so Nim's core needed generics and constraints for generics."
Does the standard library and all other user libraries work with the garbage collector turned off, or do I have to use a different ecosystem for those applications?
I agree - embedded programming is a sufficiently separate world that you'll always need a different foundation with different assumptions.
Rust which doesn't have a GC at all, but stillhas this problem to some extent. On a microcontroller you generally don't want to use the standard library with all its assumptions about heap allocations always succeeding. This is why 'nostd' and libraries like heapless exist.
I agree, but there is a big difference between not being able to use almost any library, and being able to use most libraries.
There are many many libraries on embedded Rust, and my embedded and not embedded projects do share many libraries.
I have audio programming in mind. Not that you can't do audio programming in GC-enabled languages, it's just that's it's quite frown upon in this circle (for good reasons). I'm sure there are workarounds though.
Here is an example of a super collider plugin written entirely in Nim with the GC turned off: https://forum.nim-lang.org/t/3625
The author seemed to find the experience very pleasant, and the performance was great.
A related question is whether you can disable GC on a thread by thread basis.
Interesting, I wouldn't even thought someone could consider it a criterium for a language choice. Now I wonder if most people care about it like you do.
https://en.wikipedia.org/wiki/Yukihiro_Matsumoto (see the first paragraph)
A huge part of the reason that people love Ruby even when there's so many Elixirs/Elms/Crystals/a hundred more around is simple: it's a community full of nice, smart, interesting people with varied interests that extend beyond coding.
Isn't everyone describing the community they belong as such? I have yet to meet someone advertising his programming language of choice with “the community is full of morons, join us!”…
One thing that is frustrating for anyone hearing about Nim for the first time is that it's really hard to look at what appears to be yet another slightly different take on Rust or Go and intuitively understand why it exists.
There is absolutely a grid that can be populated: who started this, is it corporately affiliated, what languages is it similar to, what is the motivation of its creators, what is the philosophy driving its design, is it intended to be an eventual competitor or replacement for languages XYZ, where does it sit on the spectrum between raw assembly and MS Access, is it intended to be a "systems language" eg can you write a kernel or device driver using it, does it require a VM, is there corporate support available, how established is the developer community, are there books and courses, does it have garbage collection, is it a compiled language, is it a strongly typed language, what are the three perfect use cases, how stable is the API, what's the package management story, does it have a reasonable standard library, is there editor support for color coding and linting, what's the job market look like, is the documentation well-written, and most importantly, if we all got in the same room, would I want to hang out with the people excited about this language?
The cost of learning yet another language vs actually spending your finite spare time building something (or with your loved ones, or reading, or sleeping, or making art) is insanely huge.