News for hackers. Stuff that matters.

FabCH

Partially.

The default git diff uses the Myers diff algorithm as implemented in xdiff. It's a line based diff with some pre and post processing for better results.

The move detection is a post-processing step that only works on lines and additionally only works if you also enable coloured output.

It can't match "moved-but-eddited" code and it can't match "this one log line was copied 3 times in the new code" or the refactoring equivalent "these 2 blocks of code were merged into one reusable function".

You have to use more syntax-aware algorithms to get true move detection. OmniDiff matches the nodes of the AST to get some of that syntax awareness.

Depends on what exactly you mean by word files and what exact format is used.

It will diff OpenOffice compatible files.

Ok I had a deeper look at twain now.

If I understand your goal correctly, you are looking to especially focus on the visual part of the diff and provide a nice UI for it, and you would like to use OmniDiff as a library?

If so, then let's talk. I have been delaying the design of the API because nobody has a use for it yet, so it is mostly driven by the needs of the CLI/TUI at the moment.

Ping me on my email if you want.

Feel free to subscribe to https://github.com/ivankovic/omnidiff/issues/62 to get notified when Lazygit support is added.

An extremely thorough comparison against difftastic is in the research/ directory. I don't recommend reading it, unless you are really into diffs. Even describing the dataset takes a good half an hour.

Multi-file support is on the roadmap. Subscribe here: https://github.com/ivankovic/omnidiff/issues/63

https://github.com/ivankovic/omnidiff/issues/61 feel free to subscribe to the issue to see when it is done.
Unfortunately that is a comparison I will not be doing because BeyondCompare is paid closed-source software.
You need to maximize the size of the residual tree sent to the final APTED step.

I actually don’t know how to do that and stay under a bounded edit distance.

Alternatively, you can crash it if you just encode the entire “Star Trek - In the pale moonlight” episode as a sequence of bytes in a const int array in C++.

It’s in the research/ directory.

Fair warning: the true comparison is fairly complex and doesn’t really lend itself to “quick comparison”.

As soon as you demonstrate editing a file with less :)
Press “n“ or “p” to jump to next/previous difference.

Press ? for the help menu.

https://ivankovic.github.io/omnidiff/showcase/content.html

Note that you do need a terminal with image support, like kitty, alacritty.

If you do not have image support, it does its best with image2ascii, but it is obviously severely limited.

Comparing is slightly complicated but roughly speaking:

Difftastic only diffs code, which is only 2/3 of all diffs.

When only measured on stuff difftastic supports:

Difftastic is perfect on 64% of diffs and near-perfect on 17% more.

Omnidiff is perfect on 90% of diffs and near-perfect on 5% more.

Speed is

p50 - 35ms difftastic vs 4ms OmniDiff p99 - 637ms vs 420ms

Difftastic is MIT vs AGPL.

As long as they open source all modifications and follow AGPL to the letter so that anyone can self-host the version they are selling, I’m ok with that.

AGPL totally allows commercial use. You just have to give back.

The CH in FabCH stands for Switzerland :)

… we don’t know what to do with AI copyright either. I assume a referendum will sort it out at some point.

I’m perfectly happy to defend the copyright in the EU/EFTA and CAN/US.

If it’s a small company, it’s not worth it, but then again they are not causing a lot of damage either. If it’s a big company, they will find me unexpectedly resistant to usual legal bullying tactics. I have legal insurance, multiple lawyers as friends and don’t have a job so have loads of time for filing paperwork.

Don't feel dumb. You are just an outlier on the visual preferences.

I actually do TWO solutions for each test case: a minimal diff and a maximal diff. Pressing 'm' in the TUI pops up a settings page where you can choose how the diff works.

I am actively researching how many ways there are to "paint" the same diff and user preferences and non-uniqueness of diffs is something I am super interested in.

If you would like to, it would be super useful for me to know what diff you don't like and why. It would help make the diff better for users with different visual preferences.

It's a tradeoff. The length itself is not a problem. It will happily diff normal multi-MB files.

The problem is the degenerative syntax tree that TreeSitter produces in case of, for example a multi-million array of bytes. This causes a stack overflow in the TreeSitter C code which takes down the Rust process with it.

The correct solution would be to prevent TS from crashing, or isolate the C<->Rust boundary better.

Or just, not support that for v0.3. Because if you are diffing the collected works of Shakespeare as a binary blob in a C++ header, maybe diffing isn't your biggest problem.

The crash will happen in TreeSitter which is a widely used AST library that all modern editors, including neovim, depend on for syntax highlighting and which is also used by many LSPs.

Additionally, here's an appropriate XKCD: https://xkcd.com/378/

Good point.

Thanks!

Perhaps the terminal is too small? What happens when you hit tab or “v”?
It’s a 5 step pipeline.

In practice, tree hashing is doing a huge amount of work because most diffs are tiny. Half of all diffs are less than 10 lines, so a hash can extremely quickly match large subtrees.

In a profoundly degenerate case of a C++ header file with only a 20MB array of bytes, it detects a degenerate tree and uses Myers O(N) sequence diffing algorithm instead of treating it as a tree.

It will crash.

But so will every other tool, because it is things like a C++ header file with 128MB of PNG data hardcoded as an array.

Editors don’t even open those files.

Depending on how proprietary their formats are, it could be done.
At the moment the algorithm for code diffing is a particularly efficient implementation of APTED which is a tree diffing algorithm that doesn’t natively construct anything useful for a three way diff. I have a notebook with a bunch of failed attempts at a good algorithm for a three way merge, but so far nothing particularly impressive.

It’s a long term goal to solve, but it’s the hardest one.

I’ll definitely reach out to you to upstream. Thank you for the kind offer!

OmniDiff is character based by default. It will only highlight the changed characters. I suppose I could add an option to additionally highlight the line as well. Thanks for the feedback!

As for packaging. Unfortunately AUR is closed at the moment due to spam. I can add brew commands for Linux, happily.

In terms of “better”. It’s actually somewhat complex, but a simple way to think about it is that if you move a function up or down in the same class, text-based diffs paint that as “Deleted here, Inserted there” where ideally you would want it to say “Moved”.

It was a tongue-in-cheek joke :)

But it was worth pointing out the difference in license, it is one of the ways in which the projects differ.

It should… it considers wide to be 200+ columns.

You can force it with “v”.

I am not super set on that value, if the community has a better default for “wide”, happy to modify it.

If an individual went through all this effort just to release it under MIT, I’d honestly be mostly just surprised someone cared so much. I’d probably just send them my own project relicensed under MIT out of pure curiosity.

But I am counting on the fact that _companies_ would be too worried about legal uncertainty to try this.

Comparing is slightly complicated but roughly speaking:

Difftastic only diffs code, which is only 2/3 of all diffs.

When only measured on stuff difftastic supports:

Difftastic is perfect on 64% of diffs and near-perfect on 17% more.

Omnidiff is perfect on 90% of diffs and near-perfect on 5% more.

Speed is

p50 - 35ms difftastic vs 4ms OmniDiff p99 - 637ms vs 420ms

Difftastic wins on license. MIT vs AGPL.

Older »