- Storing the AST on the disk in a million files is not necessarily the best use of the filesystem. In contrast, most languages store text files on the disk, and build up a similar AST in memory only
- You can't view your code without special tools, which means all text editors/version control etc. need to be Unison-aware
- Since the language is append only, all edits look like additions in version control
- Their solution for the diamond problem (depending on multiple versions of the same library) is having hard dependencies on exact versions, and including both copies can be at best wasteful, at worst bad (what if v2 fixes a bug that was in the v1 dependency), I think this is a hard problem, and the reason why semver exists
- As others have mentioned, the append-only nature of the language makes bugfixes difficult
- Solutions that dynamically discover code dependencies and automatically run tests exist for both procedural and functional languages
- Detecting that 2 things are the same through hashing is nontrivial, can it detect that 1 + x + 1 is the same as x + 2? The ASTs are different
(Which is not to defend the rest of the append-only immutability of the rest of the language, that looks a bit whack -- but then I've seen whack stuff get wildly popular, so I have no idea -- but while having 2 versions loaded at the same time might be useful I'm not sure I want to deploy every version that has ever existed that smells way too bloated)
You can scream at the developers that they've violated semver but a "bugfix" is entirely subjective (relevant xkcd, spacebar heating, etc).
And even when developers violate semver in a point release the problem still exists. They actually rarely, if ever, rollback with a 1.0.2 that is equivalent to 1.0.0 and instead usually move forwards.
And if you have a language that supports loading 1.0 and 1.1 then there's no point in being artificially constrained over which two versions can be loaded at the same time based on the label, the underlying framework shouldn't be built to care. There's no need for a multi-version library loader to care about what a bugfix is.
Semver would just be an artificial impediment at this level.
It doesn't contradict SemVer.
However when you do that, it becomes a big deal to jump from one version to another even if the breaks are minor. So pros and cons.
So say there is v1 and v2 of a utility lib in my dep tree, but actually only using func A from v1 and func B from v2. Then I just have the AST of v1.A and v2.B in my deps and everything works.
A new codebase format just uses a sqlite database instead of a million files
> Since the language is append only, all edits look like additions in version control
Traditional methods of showing change in verson control, that is text diffs, don't make sense here anyway
> Detecting that 2 things are the same through hashing is nontrivial, can it detect that 1 + x + 1 is the same as x + 2? The ASTs are differen
It can't detect that. It if could it would be pretty cool, but I don't think it would improve the usability too much
Hashing was created to prevent collisions and ensure small changes have big differences in result. The first requirement makes sense here, but not sure how the second helps.
I'm not saying it is worse than nothing, but sometimes ideas have a way of sticking around too long and making people comfortable.
Unison solves the issue - there isn't any binary incompatibility, because the transitive versions of Bv1 and Bv2 cannot be in conflict - the function references are to guaranteed unique and different versions of the art.
As for bug fixes - you can specify in your code exactly which version to use.
As for editors needing to be unison aware - they just delegate everything to the compiler via lsp and bsp.
Bug fixes are no more difficult than making the change. A new version is created, and your code can now depend on it. Old code will still run off of the old version. It's up to the code owner to decide to use the new, but fixed version.
Version control is all handled in the language itself.
As for the hard hashing problem... Runar is a particularly intelligent individual. I expect that his algorithm works pretty well.
The first argument about storing the ast is moot in an age where cached compiled typescript, Python, and .class files take up inordinate amounts of disk space.
> Solutions that dynamically discover code dependencies and automatically run tests exist for both procedural and functional languages
Eh. Piping and yarn ain't got nothing on maven and ivy and apt. But yes, dependency management isn't anything new under the sun. Dynamically resolving individual function versions in packages alongside binary incompatible functions is.
One challenge I foresee is unintentional coupling. Say you have two functions:
func serialize(MyRecord) ...
func debugToString(MyRecord) ...
Now if you ever make the mistake of having giving those the same implemention, then in Unison they'd be the same hash reference, right?
Then if you want to update, say the debug print later it would update all callsites for that hash including the ones that originally called serialize(). The two are no longer distinguishable.
This is definitely an issue that is real, and is currently a problem, and that we will fix; probably by giving the function author an option to salt the hash of new definitions that have some semantic meaning beyond their implementations (appropriate for most application/business logic). No salt for definitions whose meanings are defined by their implementations (appropriate for most generic "library" functions like `List.map`).
We already make this distinction for data types, but not yet for value/function definitions.
Not only do they have to provide a salt themselves but on top of that, they need to make a judgment call of when something has "more semantic meaning beyond their implementation" (to use your words) rather than being some more "fundamental" code.
I'm also surprised that you haven't solved this problem yet: at least once a day, IDEA warns me that some portion of my code is duplicated exactly in some other area of my code, so this kind of duplicated logic is already quite common.
It's similar to how DNS can have two domains point to the same IP, but then you can change one of those domains point to a new IP.
But how do you know which name was called where if the callers referenced the content hash not the name?
If I understand Unison right, the names are used only on the developer's layer(to write code), but when you save code, it's all hash-based.
Still, Unison got my attention.
The hard part is coming up with the normalization routine which guarantees that (lambda (a) b a) == (lambda (b) a b) and coming up with the rules for statement reordering for top level and internal definitions so that you can identify semantically equivalent statements where the outcome is order invariant. This is critical for making the hash functions useful and I suspect preventing denial of service attacks on the human brains that have to audit the code.
Being able to write a version of the code and then do the equivalent of creating a package.lock file to crystallize the hashes seems like a reasonable workflow. This probably winds up being easier in common lisp though since you can put the crystallized implementations in their own packages.
You could also view this as a kind of extreme type theory where every function (with regular names) has the type of its normalized representation (compacted to a hash for sanity's sake) and then you can run the checker to see if the types/hashes have changed. If you have somewhere that keeps track of every hash that a function with a particular name has had then you could automatically refactor, or could even support having multiple versions of the function with the same name used in a program at the same time. I'm not sure how users would feel about having to carry around `(funcall ((with-norm-id '(lambda (+ a b)) f)) a b)` though ... probably just give up on editing the textual representation and go back to the image based approach of Smalltalk and Interlisp where you can hide the hashes.
Will be interesting to see how Unison evolves.
Sound a lot like darklang.
Like others, I am dubious about this being in any way a useful feature. Separating implementation from name (/interface) and binding to that interface/name instead of the implementation is one of the fundamental and useful parts of abstraction.
The first thing people do is check in textual representations of those things in version control and operate on that instead.
But then when it comes to ideas like this we just tend to say "we're trading one set of problems or another", as if we can't evaluate the problems in a similar manner. And I'm not picking on you here, I tend to do the same!
Yes, we're trading one set of problems for another, but what if the old set of problems was "O(n²)" and the new set of problems is "O(nlog(n))"? Or maybe it's the other way around. Why isn't it obviour how to apply those earlier skills here?
(0) It is easy to order two real numbers. 1 is greater than 0. But can we order points on the cartesian plane (1,2) or (2,1) or (0,100000000000)? Already at 2nd dimension not all points are easily ordered. There is solution "just give priority to 1st coordinate", but can you really completely disregard memory usage and focus solely on cpu?
(1) Theoretically, to get "best", one needs to evaluate not only O(n) of avg cpu usage, and avg memory usage, but also worst and best cases, while using knowledge of input data distribution (e.g. maybe input is almost sorted). Memory access patterns, cache locality, battery life, suitability for your hardware also must come into play. (many dimensions)
(2) Practially one has to do profiling on real hardware with real configurations / inputs / workloads. Different workloads may favor different algorithms/structures. (again results with many dimensions)
(3) Due to constrained time people will not even go over all algorithms. Real people will immediatly rule out really bad ones, then pick 1 or 2 algorithms that theoretically are good enough match, and see if their profiling results fit in cpu/memory budgets. (even if budget is not on paper, but just part of intuition)
when you change a function implementation the system has to walk the callers graph backwards starting from all the places where the function was called updating all the implementations with the new hash, then callers of these with the new implementation and so on up to main (or whatever it's called).
I had a chance to implement something like this in a system that used jbpm 3 graph language (basically process X version 1 called process Y version 1 and I updated process Y to version 2). It's nontrivial especially with recursion, I'm wondering how they are dealing with that.
f: Nat -> Nat
g: Nat -> Nat
h: Nat -> Nat
h x = g (x * 2)
g x = f (x * 3)
f x = x < 0 ? 1 : h (x / 4)
f x = x < 1 ? 1 : h (x / 4)
EDIT: nevermind, recursion is a special case handled differently.
By having the linker work on hashes of implementations you eliminate that problem but create a new problem. You can no longer change the behavior of the function because you can't change the function. That means you can't suddenly change behavior that some caller is counting on, but it also means you can't fix bugs without changes in the caller.
Maybe the simplest solution is to allow the function to change to the new version, but make it easy to revert in the event that something breaks. This of course means that you can't make the names of the functions their hash (without lying, preventing the runtime from checking that hashes ways match, or modifying emitted bytecode or native code to do what you want), it has to be an orthogonal layer on top of them like types (as I mentioned elsewhere in the thread).
With a traditional programming language you couldn't do this because "send me a copy of sort()" would be met with "which sort()?". Whereas with unison every different sorting implementation would have different hash, so there would be no confusion.
and it will make so, so much more sense.
...and if you are like me you'll probably need to read this twitter thread to get the answer to your #1 question: https://twitter.com/unisonweb/status/1173942974381744134
Basically the core idea (or one of the core ideas) is instead of a function (like fib(n) which returns nth Fibonacci number) being identified by its name (fib) as is the case with most traditional languages, it's instead identified by a hash of its implementation.