News for hackers. Stuff that matters.
Re:Open-source paved the way for today’s remote software dev... by est31 on Monday November 23, 2020 @04:06AM (#25184897)
Most of the stuff in the article I already know but it's still nice to have a document that you can point folks to who aren't familiar with these things. I think one interesting point from the article is that CVS is from the 80s, SVN from the 90s, and git from the 2000s.

The past decade had no major VCS show up, but rather was characterized by the success story of git replacing everything else. Gcc's and LLVM's migration to git isn't that far in the past at all. So I wonder, as the last decade lacked a major leap in VCS technology, can VCS be considered a solved problem now?

I recently read on hn a comment saying that Linux will be the last OS. Is git the last VCS?

Re:Open-source paved the way for today’s remote software dev... by Lichtso on Monday November 23, 2020 @06:47AM (#25185726)
I think VCS is far from solved, it hasn't even started yet. So far we only have distributed VCS for sequences (mostly text / source code) and sometimes things which can be serialized unambiguously like trees and tables. But we don't have a general solution to distributed VCS.

We currently can't use distributed VCS for:

  - DAGs
  - Arbitrary graphs (Topology)
  - 3D models (Geometry)
  - Mutli-dimensional sequences: Images, Videos, etc.
  - most domain specific problems / formats
  - ... almost everything else
Re:Open-source paved the way for today’s remote software dev... by l0b0 on Monday November 23, 2020 @01:49PM (#25190146)
Realistically, there seems to be only two ways you could solve this: built-in VCS support in the software used to edit these files or exporting them to line-based text and dealing with them like we currently do. Building support for millions of formats (and their countless variants when created by different software and even versions of the same software) into the VCS system itself I strongly suspect would be futile. The VCS would end up much bigger and more complex than the OS.
Re:Open-source paved the way for today’s remote software dev... by leobakerhytch on Monday November 23, 2020 @12:29PM (#25189188)
To add to that list: abstract syntax trees.

It would take a pretty monumental effort to build, but I think a huge step forward could be made with a VCS that has knowledge of code structure, and the relationships between source files.

Large scale refactoring is such a fragile affair with git. Conceptually though, it should be possible to cleanly and concisely represent such operations as renaming functions, reordering arguments, widening type parameters, etc.

Stability of the programming language in question is of course highly desirable, if you were to build such a system. The pace of change of Go would be much more conducive to success than that of JavaScript (or TypeScript).

Re:Open-source paved the way for today’s remote software dev... by npsimons on Monday November 23, 2020 @12:50PM (#25189447)
> To add to that list: abstract syntax trees.

Are you joking? Because you can represent these as Lisp sexps in plain text.

Re:Open-source paved the way for today’s remote software dev... by npsimons on Monday November 23, 2020 @12:43PM (#25189362)
> - DAGs

> - Arbitrary graphs (Topology)

GraphViz (https://graphviz.org/) and PlantUML (https://plantuml.com/).

> - 3D models (Geometry)

My experience here is much more limited, but I'm certain there are text based versions of 3D model formats. The names of these formats just doesn't come immediately to mind, but having worked with antenna models, we'd stay with text representations whenever possible, and it worked just fine.

> - Mutli-dimensional sequences: Images, Videos, etc.

> - most domain specific problems / formats

> - ... almost everything else

This is where things get tricky, and again, I don't have experience in these domains. I can say, I've worked on projects where binary assets were tracked in Perforce. Not sure how efficient it was with diffs, but I know it worked, and probably better than git, which yes, doesn't know anything about binaries.

Note: I am not a fan of Perforce, for other reasons, but I am not blind to the fact that it handles binary versioning better than git.

Note2: I will also add, SVGs are just one image format that can be stored as text. NetPBM (https://en.wikipedia.org/wiki/Netpbm) has also been around forever.

Re:Open-source paved the way for today’s remote software dev... by tomc1985 on Monday November 23, 2020 @11:06AM (#25188194)
Store all those things as text and you can stick them in git
Re:Open-source paved the way for today’s remote software dev... by marcosdumay on Monday November 23, 2020 @11:49AM (#25188676)
People do not want to represent most of them as text.

Some of them need high amounts of compression to be viable. Others don't have any concept of locality that one can map into a single dimension. For some, locality of changes is even an anti-feature.

Re:Open-source paved the way for today’s remote software dev... by npsimons on Monday November 23, 2020 @12:32PM (#25189235)
> People do not want to represent most of them as text.

"People" don't care what things are stored as. You could make GUI tools save to GraphViz's text based format (or heck PlantUML[0]) for graphs at least and it would work just fine.

For assets which have to be truly binary, Perforce offers versioning of those. I know it's not open source and costs money, but if it's really a need, it's worth it.

[0] - https://plantuml.com/

Re:Open-source paved the way for today’s remote software dev... by Lichtso on Monday November 23, 2020 @12:40PM (#25189327)
You can do that, but you will only get some of the features (a distributed backup system with an undo function). The most important one IMO is merging and that does not work on arbitrary formats serialized to text. E.g. let two 3D artists fork a blender 3D file in GIT, both do changes and commit them. Until here everything seems fine, now they want to merge their changes ... and suddenly everything falls apart.

Edit: To clarify, if you don't need merging than you also don't need distributed VCS and centralized VCS will be sufficient. That is why I explicitly talked about distributed VCS.

Re:Open-source paved the way for today’s remote software dev... by npsimons on Monday November 23, 2020 @12:52PM (#25189470)
This sounds like an intractable problem, and putting it politely, the fault lies in the file format, not the VCS.
Re:Open-source paved the way for today’s remote software dev... by Lichtso on Monday November 23, 2020 @01:00PM (#25189569)
You are right, in that it is very hard. That is why I said that I think distributed VCS is far from being solved.

However, I think the "fault" lies neither in the VCS nor file format. It simply shows that merging requires domain knowledge and that future VCS will need to cooperate with file formats to achieve that.

Re:Open-source paved the way for today’s remote software dev... by strombofulous on Monday November 23, 2020 @11:42AM (#25188596)
You won't really get a meaningful diff from, for example, an image or video
Re:Open-source paved the way for today’s remote software dev... by bhaak on Monday November 23, 2020 @07:13AM (#25185886)
Git has a really simple view of changes. That is, none. It just saves blobs. It doesn't really know what you commit. For example it doesn't know if commits are independent of each other.

Darcs was also from the 2000s but unfortunately not fast enough to compete with git.

If you want to see what a post-git VCS might look like, read this introduction to pijul, a VCS that is based on patches and not on blobs like git:

https://jneem.github.io/merging/

Re:Open-source paved the way for today’s remote software dev... by l0b0 on Monday November 23, 2020 @01:58PM (#25190253)
I suspect the theory behind Pijul is where VCS development goes from being a practice (build a pragmatic solution to everyday problems and iterate) to having a solid theoretical foundation. It'll be exciting to see how it evolves from here.
Re:Open-source paved the way for today’s remote software dev... by globular-toast on Monday November 23, 2020 @04:39AM (#25185070)
Git is a funny one. I do believe the data model at the heart of git has a good change at being the last VCS data model. Apart from little details like the choice of hash function, there is little wrong with it. Git's success despite its perception of being difficult to learn and use is a testament to that.

But we are long overdue a better porcelain for git. The standard CLI it comes with is just not good. Its level of abstraction is too high in a few places where it tries to make things seem not-too-dissimilar to "old world" VCS systems. And its too low in other places, for example tags are a great, flexible tool but the vast majority of users don't use them or need them, they just need to mark releases. The fact I need to write something like "git tag --annotate --sign" instead of just "git release" is a silly.

I think a concerted effort to take what git is and what we do with (and forget old VCS systems) and reinvent the interface will create the closest to a successor to git. The only porcelain that is vaguely on the right track here is magit.

Re:Open-source paved the way for today’s remote software dev... by pabs3 on Monday November 23, 2020 @05:03AM (#25185173)
There are a few things wrong with the git data model, one is lack of chunking, which makes versioning large files space inefficient. Some backup systems like borg and restic have fixed that issue in the git data model, but suffer from lack of chunking in directory listings, I assume git would have that issue too.
Re:Open-source paved the way for today’s remote software dev... by globular-toast on Monday November 23, 2020 @06:48AM (#25185735)
Maybe it's better to say git could be the last SCM then. A VCS is much more general and git certainly won't be the last version control system for things that aren't source code.
Re:Open-source paved the way for today’s remote software dev... by nix23 on Monday November 23, 2020 @06:53AM (#25185770)
>A VCS is much more general and git certainly won't be the last version control system for things that aren't source code.

Truer words have never been spoken!!

Re:Open-source paved the way for today’s remote software dev... by rakoo on Monday November 23, 2020 @05:10AM (#25185215)
Actually if you want to give credits were it is due, you should mention bup (https://github.com/bup/bup) that arguably came up with the idea of content-defined-chunking for backups, and even stores it in git.
Re:Open-source paved the way for today’s remote software dev... by stevesimmons on Monday November 23, 2020 @06:40AM (#25185696)
Some nice references here are:

* Using Rabin fingerprinting to break files into chunks: background here on SO, including link to Rabin's 1981 paper: https://cs.stackexchange.com/questions/82150/how-does-rabin-...

* Andrew Tridgell's PhD thesis on the rsync algorithm (1999): https://rsync.samba.org/~tridge/phd_thesis.pdf

* Graham Keeling's Masters thesis on content-defined chunking used in his Burp backup program (2014): https://burp.grke.org/images/burp2-report.pdf. Burp dates back to 2011 at least. Section 6 of the thesis has an interesting contemporary comparison of rsync, bup, burp, bacula, rdiff-backup and a few others.

* Bup's design docs: https://raw.githubusercontent.com/bup/bup/master/DESIGN

* A Linux Weekly News review of Bup from 2010, with comments on strengths and weaknesses of the other approaches then available: https://lwn.net/Articles/380983/

Digging these out was a trip down memory lane. I tried most of them at various times, and now use borg-backup (https://borgbackup.readthedocs.io).

Re:Open-source paved the way for today’s remote software dev... by bryanrasmussen on Monday November 23, 2020 @04:12AM (#25184929)
if the same guy made the last OS and the last VCS, that would be some achievement.
Re:Open-source paved the way for today’s remote software dev... by sammorrowdrums on Monday November 23, 2020 @04:30AM (#25185026)
If anything I'm not sure Linus is held in enough esteem outside of tech circles.

I wish all the time that more of the people I know outside of software could benefit from something like Git. It's unbelievable how powerful a tool it is, and when I watch people doing awkward and error prone simultaneous-editing and remote synching etc, it just doesn't compare.

But then git is too abstract for many. Your files are a representation of a state of combined changes, the files themselves are not the owners of their content.

I even had to have a huge debate with a developer once about how storing a git repo in Dropbox folder so they could have same exact state across two machines was a terrible idea, and already solved with Git itself...

And then how few people also know how many places the Linux kernel or derivatives live, or even that many things they use are running something based on Linux.

What an incomprehensibly huge achievement on both counts!

[Edit] typo

Re:Open-source paved the way for today’s remote software dev... by nix23 on Monday November 23, 2020 @06:51AM (#25185755)
>What an incomprehensibly huge achievement on both counts!

He is an amazing dev, don't get me wrong. BUT he had lots of luck starting from not choosing the name Freax, choosing the same license as his compiler (GPL), and a professor that did no understood free-software (Tannenbaum and his minix), the legal battle of the BSD's...and maybe a too expensive and ~slow solaris, and a really good but proprietary VCS (Bitkeeper) as a idea-blueprint for git.

Re:Open-source paved the way for today’s remote software dev... by sammorrowdrums on Monday November 23, 2020 @06:58AM (#25185795)
True, although when you compare for example to people behind large brands with frequently huge acclaim, and then glimpse inside their (frequently) empty shell that is a marketing PR machine where all the real innovation, manufacturer and logistics is all bought, I think it's ok to give acclaim to somebody lucky, with prior art and inspiration from people who's shoulders they stand on, simply for the outcome that emerged.
Re:Open-source paved the way for today’s remote software dev... by quesera on Monday November 23, 2020 @09:59AM (#25187345)
You're not wrong.

But, tens of thousands of other people had all of those same lucky things.

Linus has claimed to have been unaware of the existence of BSD, so perhaps he had a bit of fortunate ignorance as well.

(FWIW, neither Linux nor git are my preferred options, but I still think Linus has spawned great things. :)

Re:Open-source paved the way for today’s remote software dev... by nix23 on Monday November 23, 2020 @11:41AM (#25188587)
> You're not wrong.

https://news.ycombinator.com/item?id=8641073

https://groups.google.com/forum/#!msg/comp.os.minix/dlNtH7RR...

>But, tens of thousands of other people had all of those same lucky things.

Yes AND?

>but I still think Linus has spawned great things.

I did not say anything else.

Re:Open-source paved the way for today’s remote software dev... by quesera on Monday November 23, 2020 @02:10PM (#25190417)
What we call "luck" is very unevenly distributed in the world.

Linus is also lucky that he had decent nutrition in his childhood, and that Finland was not at war with Russia during his university years. He had the luck to be born to parents that were able to prioritize education over subsistence. So yes, he was lucky.

But in your previous comment, you also call Linus's choices "luck". That doesn't read fairly to me.

He had the luck to make good choices? Maybe in retrospect, but you must acknowledge that there's much more than luck involved.

Of the tens of thousands of people with enough "luck" to arrive at roughly the same place in 1991 -- by which I mean smart, well-fed, decent academic/life schedule with adequate free time, few or manageable commitments like spouses/dependent children or parents, broad and deep exposure to Unix, access to USENET and the Internet -- Linus remains exceptional.

My closing parenthetical was not a counterpoint to your message. It was to frame my perspective: I don't worship, but I do respect.

Re:Open-source paved the way for today’s remote software dev... by nix23 on Monday November 23, 2020 @06:40PM (#25193305)
>Of the tens of thousands of people with enough "luck" to arrive at roughly the same place in 1991

There where not "tens of thousands" peoples who made a ~unix kernel licensed with the GPL.

>Linus is also lucky that he had decent nutrition in his childhood

Yeah let's stop here it getting stupid.

Re:Open-source paved the way for today’s remote software dev... by quesera on Monday November 23, 2020 @07:47PM (#25193895)
Perhaps I'm not communicating well today. I apologize if so.

Among the tens of thousands who were in the right place, single-digit dozens made the attempt, and only Linus succeeded in such a visible way. If you want to argue that he was merely the lucky one among dozens, then you still have a lot of extraordinary evidence to present, but you might have an interesting story to tell.

Your original comment reads like a dismissal because Linus drew inspiration from the world he lived in, and was just "lucky".

The point is that he did the work, he attracted the community, he managed the project. None of that was luck. And none of that is highly correlated with being an "amazing dev". Of course he stood on the shoulders of giants. Of course he learned from (and drew from) the environment around him. Of course Linux would be a forgotten academic project without the GNU userland (and license, maybe). Etc etc.

We all lived in that soup. Linus did the hard work and made something from it. I thought your comment was highly uncharitable in its disregard for all of things that made Linux successful.

> Yeah let's stop here it getting stupid.

On this, we can agree.

Re:Open-source paved the way for today’s remote software dev... by globular-toast on Monday November 23, 2020 @05:10AM (#25185213)
Git is at a vastly different level of abstraction than most of the population are used to (or capable of). When Linus made git, concepts like references, pointers, indirection, graphs, hashing, indexing etc. were already as obvious to him as the sky being above the ground. Even some programmers struggle to operate at this level. It does hurt to see non-technical people inventing their own terrible VCSs, though ("Copy of Important Document 2020-11-21 (2) (Updated PB).docx")
Re:Open-source paved the way for today’s remote software dev... by ghaff on Monday November 23, 2020 @11:44AM (#25188629)
>It does hurt to see non-technical people inventing their own terrible VCSs, though ("Copy of Important Document 2020-11-21 (2) (Updated PB).docx")

To be honest, for most purposes, shared docs do revision control well enough for the purposes of text docs and presentations. I've used GitHub for some larger collaborations, but it's really overkill for most day-to-day docs.

Re:Open-source paved the way for today’s remote software dev... by bryanrasmussen on Monday November 23, 2020 @09:09AM (#25186881)
>It does hurt to see non-technical people inventing their own terrible VCSs, though ("Copy of Important Document 2020-11-21 (2) (Updated PB).docx")

I don't think that is necessarily due to not be able to use git (as in not competent to use it) but not able to use it due to restrictions in their workplace or even just unfamiliarity with the possibility of such a tool existing.

Re:Open-source paved the way for today’s remote software dev... by simias on Monday November 23, 2020 @09:31AM (#25187081)
Well that and out of the box git isn't very user friendly, even for developers who are used to dealing with command line tools. To this day in my experience most devs have a relatively shallow understanding of git and often need assistance for anything more complex than commit and push. I can't count the number of times I've had to assist a colleague for what ended up being a relatively simple merge.

I think you'd need an extremely good GUI if you wanted to market git to the masses.

Re:Open-source paved the way for today’s remote software dev... by sammorrowdrums on Monday November 23, 2020 @10:47AM (#25187958)
Hahaha, watch as colleagues stare blankly when you ask them to look at the reflog after botching a rebase and force pushing... and loads of other tools like bisect and blame heavily under-utilised, custom git hooks... The list of under-utilised git commands is long, and while perhaps a sign of a poor UX, if you use the features it really is rare that you are surprised or that you cannot return a repo to a good state, regardless of what you do.
Re:Open-source paved the way for today’s remote software dev... by simias on Monday November 23, 2020 @10:56AM (#25188069)
I do think that git's UI is a good case study on how not to design a command line interface. It's improved over the years but I remember that a decade ago the contrast was stark if you compared, say, mercurial to git.

Git always felt more like you were exposed a raw low level API on the command line with a few shell macros on top to make it a bit more palatable which is effectively what it was for a long time. I suppose it fits well within the unix philosophy of clunky but very flexible and not very opinionated tools.

Re:Open-source paved the way for today’s remote software dev... by throwaway201103 on Monday November 23, 2020 @12:18PM (#25189046)
I'm a programmer, I understand all those things well, but I struggle to use git beyond pull, commit, push.

Every time I do something more than that I end up needing to google error messages, and usually end up copying my changes to a temp directory, doing a hard reset, and starting over.

Re:Open-source paved the way for today’s remote software dev... by blackrock on Tuesday November 24, 2020 @08:03AM (#25198050)
I think he chose the wrong verbs.

pull should’ve been clone.

commit should’ve been snap.

push should’ve been commit.

This way, clone (instead of pull) will make a local copy on your computer.

This is your local working copy. When you are ready to save, you can “snap” shot it, and leave a little message. And you can do as much of this as you want.

Then when you’re ready to merge it back to the primary database, then you can “commit” it.

The whole concept of pull/commit/push seems nonsensical to me. As a layman, I can’t understand the importance of pull vs. push in this context. And this makes it difficult to explain to someone non-technical.

Re:Open-source paved the way for today’s remote software dev... by globular-toast on Monday November 23, 2020 @02:00PM (#25190277)
I assume you haven't studied git at the DAG level then. It makes a lot more sense once you do.
Re:Open-source paved the way for today’s remote software dev... by blackrock on Tuesday November 24, 2020 @07:52AM (#25197981)
Can you actually use git with proprietary binary formatted files like docx?
Re:Open-source paved the way for today’s remote software dev... by b0rsuk on Monday November 23, 2020 @04:31AM (#25185029)
It's really a shame so few people recognize him. You don't count unless you sit on a large pile of money.
Re:Open-source paved the way for today’s remote software dev... by WJW on Monday November 23, 2020 @07:50AM (#25186191)
While I like Linus as much as anyone, he had a large impact in a small field. Even then both linux and git are community projects first and foremost, Torvalds is just the BDFL but the projects would continue very well without him.

Almost nobody can name influential luminaries outside their field. Who outside the field of fiber communications has ever heard of Kao and Hockham? How many HN commenters could tell what Reynold Johnsons' team invented without having to google it? Those are just some examples from software-adjacent fields that we all use every day. Imagine how many fields there are out there that you'd never think of.

In any case, what you mention as "counting" is a very narrow approach. If you mean "has to appear before congress for questioning" then having a lot of money helps. There are hundreds if not thousands people doing impactful work that you will never read about. There is not enough time in the day to keep up with all the work done in modern society but that does not make their work "count" any less.

Re:Open-source paved the way for today’s remote software dev... by nix23 on Monday November 23, 2020 @06:42AM (#25185704)
>last OS and the last VCS

I don't think that's true, he made a kernel NOT an OS, and the last VCS...we will see. There is zero proof that any software survives forever...well ok, COBOL maybe.