Let's just keep in mind that the entire reason languages with immutable data are trending is because we already have all those languages with mutable data to use where they are fit.

A new language today shouldn't really implement Python metaprogramming, because Python already does that.

Yes, and I think the reason why Nim got so much attention was because it filled the niche of metaprogramming in a statically-typed, AOT language, which is probably the last frontier of mutability and metaprogramming. It's probably the last major new language with metaprogramming front and center we'll see for a very long time.

Honestly, I don't think I'll ever create my own general-purpose language precisely _because_ I already know languages that are perfectly tailored to my use. If I start thinking of "what would my dream language look like", I start mentally re-creating Python. And sometimes, I might be in a different mood, and I'll think "wait, I'll make a statically-typed, AOT version of Python that I can use as a systems programming language instead of C/C++", and I end up mentally re-creating Nim, even to the point of going "well, I'll borrow the static typing part from Modula-3 because Modula-3 is cool and Python has a lot of Modula-3 in it already...", which is basically what Nim did.

Indeed. That may mean fewer opportunities for "meta-programming" if by meta-programming we mean mutation of core structures. It could also be a good chance to turn around the default of declaring variables to declaring constants instead. Complex data structures would still have to be mutable by default I guess, but they would be the obvious next target to becoming immutable by default.

Why would your complex data structures be mutable? Wouldn't immutability pay off especially in complex cases?

If you make my huge NumPy arrays immutable, I will be very unhappy (unless I am significantly misunderstanding how this everything-is-immutable strategy works).

There might be a lot to be said for rust-style immutable-by-default.

First, you'd still have optional mutability. Second, arrays are rather simple data structures, but ones where you often care about performance--small tight loops are common. Compare that to the data structure that eg holding your options and flags for the computation: you don't do much computation on that, but you have lots of invariants that you want to uphold---ie lots of code, but each bit executed roughly once, no tight loops.

I assume using some form of copy-on-write.

Algorithms on immutable data structures tend to have very bad worst case peformance.

Naïve algorithms on immutable data structurs tend to have very bad worst case performance.

And there would always be the optional mutability. Your complex data structures are seldom the ones driving performance.

Ie often you have to get the complex data structures right (lots of logic, not much computation) to set up your computations on the simple ones (eg lots of number crunching on your arrays).

Yes, it's easier to reason about correctness in immutability land and add mutability in after the fact as a compiler optimization than to remove mutability for safety as a compiler pass.

Haskell sort-of does this. Laziness can be seen as a very disciplined form of mutability.

(Haskell also allows mutability that is visible to the semantics of the program, that's where the famous monads come in.)

Honestly, I found that kind of disappointing, because one of the things I like most about Python is its metaprogramming capabilities.

I'm honestly not fond of languages like Rust that try to restrict mutability. The only one of the new breed of languages I've found that I really like is Nim.

Doesn't Nim go a bit too far in that direction, though? Mutable strings seems a bit dangerous to me – won't you have to copy them all the time to make sure they don't get modified somewhere else?

In other words you'd make it slow without the ability to allow it to be fast.

> without the ability to allow it to be fast

Practically speaking, Python is still slower than a lot of languages that use predominantly immutable data structures. I think you'd want to go to a systems programming language for raw speed anyway.

Imagine immutable dict. How many programs in Python you know would still work?

"Immutable" data structures is rather confusing name. Here it is a synonym for persistent data structure [1], which means that you operate on data structures by creating new "versions" of them, which share a lot of memory with the old versions. Parts of the program that refer to the old version see it as immutable.

[1] https://en.wikipedia.org/wiki/Persistent_data_structure

Modern radix trees[1] can be as fast as the hash maps you'd find in a language's standard library while providing additional capabilities (ie fast ordered traversal). Making them immutable and persistent[2] is easy and only adds a bit of performance overhead.

Obviously programs that rely on communicating by mutating parts of a dictionary would have to be rewritten to use an immutable, persistent map—but they could still maintain similar performance.

[1]: Namely "adaptive radix trees" as described in “The adaptive radix tree: Artful indexing for main-memory databases” by V. Leis, A. Kemper, and T. Neumann <http://www3.informatik.tu-muenchen.de/~leis/papers/ART.pdf>

[2]: A persistent adaptive radix tree in Java: https://github.com/ankurdave/part

Thanks for the radix tree reference!

On the other hand, stating "they could still maintain similar performance" for immutable dictionaries is misleading at best (you are right for dictionary sizes approaching 0). Anyone can look up benchmarks on the Internet showing real-world performance of immutable data structures and make their mind up if they actually can deliver in their particular use cases.

Mutability could be always used to boost performance on Von Neumann architecture if you know what you are doing. It's just becoming impractical due to increased complexity, pervasive security paranoia and the fact that there are very few people that can get things done right. If you want to contribute to advance your case, please create a competing computer architecture as fast as Von Neumann's, physically possible, but based on immutability.

> Mutability could be always used to boost performance (...)

For the extreme case, see the EQUIVALENCE statement in old Fortran.

Well, opt-in immutable dict would be awesome. I've found that many times my usage of set and list is more appropriate as their immutable counterparts (frozenset, tuple). Lots of times I create sets or lists with the intention that they be immutable and it would be clearer if I could specify to future me/maintainers/extenders that this should remain as initialized.

Switch to namedtuples for lightweight data structures. No regrets.

Is that true? It doesn't match many of the comments on HN around React, Om, Clojure etc...

Yes, it is slow. Immutability is allowing you to write elegant concurrent algorithms at the expense of speed and memory. Imagine you are forced to never change an array of numbers for an algorithm computing matrices, rather creating new matrices for each step and specifying numbers only during initialization. But hey, you can cache them and keep track of every single copy without any confusion now! Now imagine a non-concurrent language like JavaScript - what exactly are you going to get by using immutable structures except for the feeling you are going with the times?

Reference counting and caching are probably the only areas where immutability can help for most programs.

Immutability is in fashion nowadays due to functional programming being in fashion despite the inability to ever achieve "pure" functional programming in reality - I/O is by default "dirty" (I really hate these charged words in computing), no amount of flatmapping (i.e. monads) is going to help with that.

Everything immutable (every data structure) is a really convenient way of thinking about processing data in the abstract sense. You can think about processing your (semi) structure data (records) in terms of high level pipelines and processing steps. And, for a whole class of problems this works well.

Certainly this abstraction can stretch pretty well. Esp. when your language / framework can optimize immutable data structures under the covers (mutable and reuse.) But the abstraction is just that an abstraction for a mutable platform.

The platforms of today (VMs, operating systems, hardware, storage) are not immutable and notoriously full of side effects and undefined behavior. So these immutable language / frameworks end up being good tools at one extreme (high level data processing), bad at the other (system level languages) and vary on a case by case basis.

In today's todays platforms if you want to allocate an object in your language think of the layers and layers of side effects that will happen. 1. The allocator/GC system allocates it from some kind of pool possibly. Lots of side effects there (like operating on a pool, triggering some kind of collection / cleanup, requesting a large mapping from the OS. 2. Then at the OS level accessing mapped memory has all sorts of side effect (paging, lazy initialization, etc). 3. At the hardware levels there's lots of side effects: instruction cache, multiple layers of data caches, TLB, DMA.

In conclusion, for some kinds of problems you don't need to worry about these details and for others you can't afford to ignore them.

Thanks, that's a great answer and helps me understand.

I guess there are subsets of problems where immutability is a good idea (anything needing state, 'Undo' capabilities), neutral, and a bad idea (number crunching, data processing, 'traditional'(??) programming).

I have been wowed by the beauty of the immutable approach in web apps, but definitely subscribe to the right technique for a task. I can't imagine doing the data processing in physics using an immutable language...

Indeed it boils down to what you need to achieve. You might want to use both approaches and combine them for best performance/readability.

There are specialized high-performance structures that are mutable (e.g. disruptor in HFT trading) optimizing all the way down to CPU cache lines to achieve unbelievable speeds, yet they work nicely if the objects they refer to are immutable.

So just take all advices to switch to immutability with a grain of salt and learn to figure out which approach is suitable for what part of your system to have it both performing well and being able to conceptualize and maintain the code nicely ;-)

> Reference counting and caching are probably the only areas where immutability can help for most programs.

There is another use spurred by React: when you have a new state to represent, you have to diff it against current state to know exactly what changed and work only on this diff. "Immutable" structures make this trivial.