News for hackers. Stuff that matters.
Re:Squash your commits by phasmantistes on Friday April 01, 2016 @03:53PM (#11408008)
I actually disagree. Large teams that still have linear commit histories doesn't mean it is a lie. It means that the code review process is more important that the code writing process.

For example: I check out a repository, and create a local feature branch. I create a commit containing the tests for the new feature, then one for the first draft of the new feature, then two or three for bugfixes. Each commit is small, and self-contained, but importantly isn't standalone. If someone checked out the repository in the middle of my chain of commits, they wouldn't have a working product. Then I upload my change for code review. There's no point in reviewing each of my ~5 commits individually: they only make sense to the reviewer as a combined unit. And there's no point in landing them individually: they only make sense for the overall project history as a combined unit.

In a project with many developers (e.g. 1,000 like the Chromium project), every developer has different local practices. Some keep their work based on HEAD of master via rebase, others via merges. Some do test-driven development, some don't. Making the code review the atomic unit of work, rather than the messy string of local commits, helps the project enforce common etiquette, commit formatting, and readable history.

Re:Squash your commits by sulam on Friday April 01, 2016 @05:54PM (#11408889)
caveat: I was responsible for code review for 2000+ developers.

We only allowed squash commits on master because of what you're describing. That is the level where history "made sense". However, for code review, we wanted to support both styles, because there is an advantage sometimes to seeing the sausage being made. For instance someone will refactor something -- maybe change a method name. Then they apply that refactoring at all the call sites. Very conscientious developers would break this into two commits. We didn't want the first commit on master, but it made sense to review this way, because it was easier on the reviewers: change, effect of change on everything else.

I call this "telling a story" with your commits. There's a lot of value in that style if you have the time to do it.

The other style of commit-by-commit reviewing, where I see all of the work in progress commits, I don't find valuable at all and I _definitely_ don't want to see on master.

Re:Squash your commits by andersonvom on Friday April 01, 2016 @06:16PM (#11408980)
While this is "telling a story", the first commit will break your code for no good reason (i.e. if you rename a method, but not its usages, hell breaks loose). This make you lose one very useful features of git: the ability to binary search for the place where a bug was introduced - git bisect.
Re:Squash your commits by hinkley on Friday April 01, 2016 @07:01PM (#11409183)
But so does squashing, which is one of a handful of reasons I hate most uses of squashing.

Only squash when it removes bug that only ever existed on your machine. Everything else should be recorded in the history. Forensics are important to the long term health of your project and you impoverish yourselves by scrubbing the crime scene.

Re:Squash your commits by Pyxl101 on Friday April 01, 2016 @10:51PM (#11410007)
> Everything else should be recorded in the history. Forensics are important to the long term health of your project and you impoverish yourselves by scrubbing the crime scene.

OK. So would you support an IDE that generated one individual commit per keypress? When I type "hello", that's five individual commits each changing a single letter.

That is the true history of what happened, and it's typically recorded in your IDE's undo log in memory, such that you can step back and forward through it. Are you OK with pushing this commit history to your project? That's the true history of what happened, after all, and it could be forensically useful.

When we commit code to share it with other people, we recognize that only a certain level of detail is relevant to them. The physical sequence of letters that I pressed isn't typically relevant to someone else. Rather, the logical set of changes is what affects them.

If you agree that a single commit per letter of keypress is undesirable then you agree with me in principle. There is a finite level of detail that makes sense to share, practically, with current source control systems, and what we're arguing about is how much to share.

I feel like this is important to point out because people frequently make this argument about the "true history" of source, and all that, while neglecting the fact that the commits they choose to ship are already an arbitrary simplification of the "true history".

Re:Squash your commits by jasonkester on Saturday April 02, 2016 @02:53AM (#11410544)
So would you support an IDE that generated one individual commit per keypress?

That would actually be awesome, if the tools supported it. Imagine how easy it would be to find bugs with a bisect if it can drill down to the actual keystroke that introduced a bug.

Realistically, you'd want to back out a few notches though. Say, every time the dev hit Shift+Ctrl+B (or tabbed over to the browser or whatever signifies "build the project" in the environment in question), so that you get an indication that the current state of things was meant to at least compile and run.

But yeah, that's the value of source control. Being able to dig back in history to the exact spot where a bug was born. I can't understand all why so many people here would want to scrub that away.

Re:Squash your commits by hobs on Saturday April 02, 2016 @09:47AM (#11411492)
I am told that Google's filesystem for their dev workspace is somewhat similar to capturing every single keypress as an immutable layer forever.

Apparently introducing interesting problems when people create a log file one byte at a time, up to several hundred megs. (just 100000000 files for one little project, oops)

Re:Squash your commits by phire on Saturday April 02, 2016 @04:23AM (#11410734)
I actually implemented an eclipse plugin which recorded every keypress (among other things) for my 3rd year project at uni.
Re:Squash your commits by chris_wot on Saturday April 02, 2016 @04:16PM (#11413191)
That's pretty interesting... did you ever publish this code?
Re:Squash your commits by MaulingMonkey on Saturday April 02, 2016 @02:35AM (#11410506)
> OK. So would you support an IDE that generated one individual commit per keypress? When I type "hello", that's five individual commits each changing a single letter.

Not OP but - conceptually? Yes. I've written file formats which preserve undo/redo history, for example. The caveats:

1) I don't want to inadvertently leak my password. This is an issue with things such as local bash command history buffers as well, and unreviewed autocommits in general. You could say it's too useful - to the wrong people!

2) My tooling needs to be built around a different level of granularity as the default. I'm not OK with single letter commits cluttering up my git log, for example. Having them around to drill down into if I need them? Sure.

Per-letter detail is so granular that even undo/redo systems will often squash history states together. Observing your exact typing Cadence / the extra evidence of initial authorship is niche enough that I'm quite willing to sacrifice that level of detail for the sake of performance, maintainability, or basically any and every other excuse you can think of.

> When we commit code to share it with other people, we recognize that only a certain level of detail is relevant to them. The physical sequence of letters that I pressed isn't typically relevant to someone else. Rather, the logical set of changes is what affects them.

I have never seen a codebase with perfect commits. The ones that always give me trouble wrapping my head around are the squashed "logical set of changes", where the set size is way too damn big. Reverse engineering a saner overview from a series of tiny commits is way easier.

And the physical way something was done does matter at times. I'm going to pay way more attention to "whitespace cleanup" commit done by a human than "whitespace cleanup" commit done by vetted tools, for example. In whitespace significant languages, the former may trigger a full code review. Similar concerns with a lot of refactorings, actually.

> If you agree that a single commit per letter of keypress is undesirable then you agree with me in principle.

Per the above, I can only agree with you in practice :)

> There is a finite level of detail that makes sense to share, practically, with current source control systems, and what we're arguing about is how much to share.

Agreed. But I haven't found a single, solitary codebase, where I'd ever argue "less". I can't even recall a single, solitary commit where I'd have ever argued "less". Commit directly to mainline to fix a single character typo? I'll be annoyed if your changelist description was too terse! I want to see:

  Fix the build:  Fix typo 'baz' -> 'bar'
  Fix documentation typo: 'baz' -> 'bar'
I've seen a codebase where a majority (read literally: >50%) of the commits were "good enough". It was beautiful.

A coworker of mine shared he'd collected stats on who made the most commits/day to test a hypothesis. Due to some outliers, he'd decided against it, but thought I'd find the stats amusing. Out of ~50 people (~15 programmers), I (the most recently hired programmer) topped the chart. Second place? The build server account (thanks to nightly build scripts.) I was indeed amused.

Re:Squash your commits by chris_wot on Saturday April 02, 2016 @02:57AM (#11410550)
I'm curious, what do you think of the commit history of the last few years of LibreOffice?

https://cgit.freedesktop.org/libreoffice/core/log/

(Don't look at the OpenOffice.org years, they literally took a whole bunch if development work from SVN branches and then merged them in as a single commit and put in single line descriptions with internal tracking numbers and odd project management codes... utter disaster! And of course the branches are now all lost...)

Re:Squash your commits by MaulingMonkey on Saturday April 02, 2016 @04:16AM (#11410719)
How about I just look at the latest ~33h directly on that page ;)

I like the scope of a lot of those commits, although a few are still chunkier looking than I'd like - take that with a grain of salt, though, as I don't have a good enough feel for the codebase to reasonably estimate how much more they could be chunked up. Pretty much everything has a review link, which is nice. I'd expect more back and forth in the comments, but perhaps that's handled out-of-band.

I have lots of nitpicks with the actual changelist descriptions where I'd want things to improve. "Clean up" could mean just about anything - I must go to diff (and expand the context) to understand e.g. https://cgit.freedesktop.org/libreoffice/core/commit/?id=945... properly. I'd be inclined to instead write:

officeipcthread: Cleanup RequestHandler::Enable: Early bail, remove aDummy (just use aUserInstallPath directly), move declarations.

Now I know scope, and the types of changes (refactoring worth reviewing if looking for breakage, not just ignorable whitespace / comment changes.)

No gripes with the overall style on this one, although interacting with a security component, I'd want multiple reviewers: https://cgit.freedesktop.org/libreoffice/core/commit/?id=2a9... .

I'd lean towards linking a screenshot of at least the new version of UI when the file being modified "isn't human readable" (read: is modified with something other than a text editor, even if I can totally read it) which would apply to e.g. https://cgit.freedesktop.org/libreoffice/core/commit/?id=72c... .

EDIT: Formatting, + rationale RE: changelist description.

Re:Squash your commits by chris_wot on Saturday April 02, 2016 @06:56AM (#11411062)
Ah... sberg's one commit message that could be more descriptive :-) That NSS commit has two very experienced reviewers, but fair point! wrt screenshot linking, I guess a link to the LibreOffice bugzilla would be nice.

Thanks for the info, that's really quite insightful. Appreciate you taking the time to give it!

Re:Squash your commits by MaulingMonkey on Saturday April 02, 2016 @04:18PM (#11413205)
> That NSS commit has two very experienced reviewers,

I must be blind - somehow I only saw one the first time around, but they're clearly both there.

> Thanks for the info, that's really quite insightful. Appreciate you taking the time to give it!

I'll take your word for it ;). And no problem.

Re:Squash your commits by chris_wot on Saturday April 02, 2016 @04:21PM (#11413219)
I'd love to see a system that can show reviewers as metadata within git itself.
Re:Squash your commits by lerpa on Saturday April 02, 2016 @04:28PM (#11413257)
> My tooling needs to be built around a different level of granularity as the default. I'm not OK with single letter commits cluttering up my git log, for example. Having them around to drill down into if I need them? Sure.

And that would be nice to have, a vcs that allows you to fold and unfold commits to different levels of granularity.

Wanna have only a straight line history with all feature branches squashed? There you have it. Just need some of them like that? Sure enough. What if you want every time the file was saved? Not a problem, just configure it to commit automatically.

Re:Squash your commits by jasonlotito on Friday April 01, 2016 @10:48PM (#11409995)
> Only squash when it removes bug that only ever existed on your machine. Everything else should be recorded in the history.

You can squash into master without losing the history of the code review with git. This gives you the best of both worlds, a more accurate history than the one you propose, and a master that isn't broken.

Re:Squash your commits by Pyxl101 on Friday April 01, 2016 @10:58PM (#11410020)
I've often wondered if it would be possible to design a source control system that could track this lineage. Kind of like, "inside" of a commit would be a hidden history that you could unpack if you wanted to. "This single commit is actually the squash of these other commits made by this guy". "This commit is actually the rebase onto mainline of this commit a guy made when his machine was behind". The commit would look like normal to the current source control CLI, by default, but would reveal this hidden level of detail if you inquired "How was this commit made?" - then you'd get the reflog.

I've wondered if support for a feature like this could finally resolve the "debate" between merging and rebase. The people who like having the full history (whatever that means) would have it, and the people who care about logical changes would see it by default. The cost would be that the repo history would indeed have the full level of detail, but you could imagine supporting a cleanup operation that erases this "origin" information for commits that are sufficiently old, or else simply doesn't pull the info by default and fetches it only when necessary.

(You can kind of simulate this using merge commits, by only looking at one parent, but it doesn't really work out well in practice.)

Re:Squash your commits by chris_wot on Friday April 01, 2016 @11:54PM (#11410140)
Wouldn't this be a branch?

FWIW, the OpenOffice.org guys had this insane workflow where they squashed all commits into one commit and then out a summary of all the changes into the commit message. Then they scrubbed the private branches.

That methodology is an extreme example of where squashing does NOT make sense.

Re:Squash your commits by superuser2 on Saturday April 02, 2016 @01:14AM (#11410339)
Have you used Phabriactor? This sounds an awful lot like a Diff in its terms. "master" is a linear sequence of Diffs being landed, with Diff number in the commit message. You can paste that diff number in the web UI and see the individual commits that were part of it.
Re:Squash your commits by MaulingMonkey on Saturday April 02, 2016 @02:48AM (#11410535)
> You can squash into master without losing the history

This is otherwise known as a merge. A single atomic change to master, referencing only the final files, with the first parent referencing the previous master that was also "not" broken.

It has only one change: A second parent, referencing the more accurate history.

Re:Squash your commits by furyferret on Saturday April 02, 2016 @04:48AM (#11410788)
What you're describing is definitely a squash.
Re:Squash your commits by MaulingMonkey on Saturday April 02, 2016 @05:23AM (#11410864)
Given: http://robey.lag.net/images/no-ff-5.png

If we start without M, and start with master=R2, temporary-feature-branch=R5, I expect the following outcomes:

"squash" will leave us with master=M, and the graph R1 -> R2 -> M. Once temporary-feature-branch is deleted, R3-R5 are forever lost.

"merge" will leave us with master=M, and the graph R1 -> R2 -> M, + R2 -> R3 -> R4 -> R5 -> M

"rebase" will leave us with master=R5, and the graph R1 -> R2 -> R3 -> R4 -> R5. Once temporary-feature-branch is deleted,

The first case loses the history of the code, and thus can't be what we're talking about.

If you call the second case "squash" I find that very confusing. It's also not what github is referring to when talking about "squash your commits" in the parent article, they're talking about the first case. Now, git confuses the issue a bit too - perhaps I should be labeling the second case a "non-fast-forward merge", but I'd still find that clearer than "squash without losing the history."

Re:Squash your commits by junke on Friday April 01, 2016 @08:28PM (#11409569)
That's why you can "skip" a commit during git-bisect[0].

I had to find when a bug was introduced, and guess what? it was inside a single massive commit touching hundreds of files. I prefer atomic commits with good messages, thanks.

[0] https://git-scm.com/docs/git-bisect

Re:Squash your commits by voltagex_ on Saturday April 02, 2016 @01:45AM (#11410412)
Yeah, I wonder if I could write a checkin policy for to enforce "small" changes without it becoming too annoying.
Re:Squash your commits by sulam on Friday April 01, 2016 @06:46PM (#11409121)
Which is why that commit doesn't get on master.
Re:Squash your commits by phasmantistes on Friday April 01, 2016 @06:13PM (#11408968)
Yep, in chromium-land we do something similar:

You can upload multiple different versions of a single code review, and reviewers can diff both against the base and against previous versions of the review. This is helpful for showing "stories", responses to comments, and for "my original commit got reverted, so here I've reuploaded it, and then also uploaded the fix, so you can clearly see what's different this time".

Re:Squash your commits by chris_wot on Saturday April 02, 2016 @03:10AM (#11410581)
Is that using gerrit?
Re:Squash your commits by sulam on Saturday April 02, 2016 @11:30AM (#11411852)
Rietveld looks like. Which used to be the code review tool at Google, but has since been replaced. Not by Gerrit. ;)
Re:Squash your commits by tedmiston on Friday April 01, 2016 @06:08PM (#11408944)
> The other style of commit-by-commit reviewing, where I see all of the work in progress commits, I don't find valuable at all and I _definitely_ don't want to see on master.

This is the style I use for code reviews.

There's not a lot of tooling to support it, at least with GitLab, but git-playback [1] is kind of interesting.

[1]: http://codingfearlessly.com/2012/05/14/git-playback/

Re:Squash your commits by vcarl on Friday April 01, 2016 @08:23PM (#11409551)
I try to shoot for meaningful individual commits, squashing to get the intent of each change (more or less) self contained. I've found it really helpful if I ever need to `git bisect` to find what introduced a bug, I get a small(ish) commit that only affects a single module. Very helpful when debugging code I wrote last year.
Re:Squash your commits by radarsat1 on Saturday April 02, 2016 @12:10PM (#11412025)
In that sense it would be cool if git supported something more like "branch collapsing," which could be used instead of squashing a branch and rebasing that on master. It should be possible to "view" a branch as if it were a single commit on master, with its own comments etc (from the merge commit perhaps) and then expand it. Perhaps this is just a UI problem..
Re:Squash your commits by WorldMaker on Monday April 04, 2016 @11:24AM (#11422581)
I do think this is mostly a UI problem. As someone else suggested in this thread sites like GitHub which show only a linear list of commits should probably default to something more like `git log --first-parent`.

There also sounds like a need for `git bisect` to be smarter about DAG traversal, in that it seems to treat the branch as if it were a flattened linear list rather than taking advantage of the merge structure of a branch. This also sounds like something that should be relatively easy to make smarter.

Re:Squash your commits by suls on Saturday April 02, 2016 @03:11AM (#11410583)
How did you - on that 2k+ developers project - deal with the fact that sometimes someone is working on a change in a feature branch (a) that the an other developer also needs for her feature branch (b)? Would you move the commits that need to be shared away from the features onto a seperate branch (c) and then .. ?

I am really curious how you handled these sort of situations at that scale!

Re:Squash your commits by jldugger on Friday April 01, 2016 @03:58PM (#11408043)
> It means that the code review process is more important that the code writing process.

If that were true, the optimal solution is PRs with individual commits that all pass testing. I find it much easier to review a series of small changes for logical correctness than mashing them together into a single PR. Github recently added this as a feature, so I'm not in a completely invisible minority there.

And then, when the review is over, having discreet commits makes git bisecting down to the commit that broke the system more granular.

Re:Squash your commits by phasmantistes on Friday April 01, 2016 @04:03PM (#11408082)
Absolutely. If someone formulates their PR such that every commit in the chain is small, easily reviewable, and passes all tests, that's fantastic! That makes reviewing code, searching history, and bisecting all easier.

Unfortunately, that's not the 90% case that I see. Most of the time a multi-commit PR contains N-1 commits of incremental development and one final one that fixes all the tests and typos and removes debugging print statements. Neither the project nor the author benefit from having those intermediate commits integrated verbatim.

Re:Squash your commits by jarfil on Friday April 01, 2016 @05:32PM (#11408757)
If people generate N-1 commits with a final one to clean things up, maybe people should learn about git stash and making some WIP branches, then squashing commits themselves or better yet, keeping their own history clean, instead of submitting PRs full of crap.

I know, it might be too much to ask of people... oh well.

Re:Squash your commits by hinkley on Friday April 01, 2016 @07:06PM (#11409202)
The Mikado method is sadly underemphasized.

Just because you smash away at code for hours doesn't mean that's how your commit history should look. You can revise history in ways that are beneficial instead of destructive.

Re:Squash your commits by cortesoft on Friday April 01, 2016 @08:31PM (#11409585)
What is the cost of just doing it at the last merge step? Makes it easier, doesn't it?
Re:Squash your commits by chris_wot on Saturday April 02, 2016 @03:12AM (#11410589)
Yeah, but dreadful if you try to bisect a problem.
Re:Squash your commits by rjayatilleka on Saturday April 02, 2016 @09:43AM (#11411476)
Yeah, and that's why you squash all the intermediate commits first. If all your commits on master are useful and meaningful individually, bisect works great.
Re:Squash your commits by forrestthewoods on Saturday April 02, 2016 @03:59AM (#11410692)
> Neither the project nor the author benefit from having those intermediate commits integrated verbatim.

Sure they benefit. You can see the thought process that went into the commit. If there's something that seems weird or out of place you can see how it evolved into existence. That can be exceptionally useful.

Re:Squash your commits by voidlogic on Friday April 01, 2016 @05:06PM (#11408612)
>If that were true, the optimal solution is PRs with individual commits that all pass testing. I find it much easier to review a series of small changes for logical correctness than mashing them together into a single PR.

This is too myopic and explains how you can end up with good code, but bad architecture. A good review ensures both.

I like my PRs to be about high level goals, and I want them made of lots of commits I can review. The commits themselves can be the result of re-basing (which is fine within feature branches) and maybe not truly chronological.

Re:Squash your commits by stingraycharles on Friday April 01, 2016 @04:11PM (#11408148)
I agree with what you're saying, good code review and good code writing aren't mutually exclusive. However, the truth of the matter is that code review is a far more clearly defined moment in a workflow than writing good code (which is more of a habit than an actual action). As such, it is far more efficient for a team to accept reality and squash after PRs (which is now first possible), instead of relying and keep correcting all your coworkers when they don't properly squash commits before pushing.
Re:Squash your commits by WorldMaker on Friday April 01, 2016 @04:34PM (#11408369)
«Making the code review the atomic unit of work, rather than the messy string of local commits, helps the project enforce common etiquette, commit formatting, and readable history.»

I agree that code reviews should be important, and key to understanding the history of a project at a more useful timescale. I just disagree that they should be "atomic" and that a reviewer (even, or especially, a later code archeologist) may not have reason to inspect or dive into smaller units within code reviews.

Where I think that we may agree is that I feel that even if they shouldn't necessarily be "atomic", I agree that code reviews should probably be first-class objects when talking about and dealing with source control. In git, you can use --no-ff merges today as a useful approximation of code review boundaries (especially with PRs and GitHub's default --no-ff and including linking PR #s). It might be nice to see code reviews or other aggregates of commits/commit graphs be truly first-class citizens of git in some manner.

Re:Squash your commits by procrastitron on Saturday April 02, 2016 @03:12AM (#11410592)
I agree with you, and I prefer the --no-ff merge option for submitting reviews, but I think the argument for it would be a lot stronger if 'git bisect' supported a --first-parent flag the same way that 'git log' does.
Re:Squash your commits by lotyrin on Friday April 01, 2016 @04:35PM (#11408375)
Yep, the key to understanding git (why there's mutable history, etc.) is to understand that because of its history (designed by Linus) it's not designed to make day-to-day developers's lives easier, but to serve foss project maintainers and release engineers.

People who review code for inclusion in a project, want to track meta-progress on issues, want to pin versions for release, etc., mutable history means they can squash fixups or fix your whitespace for you, rebase changesets onto other changesets, have history that reflects the project management strategy, etc.

Re:Squash your commits by Justsignedup on Friday April 01, 2016 @04:56PM (#11408553)
Also it is very likely that my feature branch isn't really complete during all the little commits I make. The commits are useful to me, but don't often indicate intent too much. The whole thing put together may be easier to diff when a few years in the future someone tries to figure out what happened there.
Re:Squash your commits by itaysk on Saturday April 02, 2016 @03:42AM (#11410654)
Regarding your example - why would you commit any of the 5 little commits, If they didn't complete the task, and also are breaking the product? wouldn't it be better to commit only when you completed the task you were working on? (I mean, if the task is made out of sub-tasks, then I would also commit each subtask or milestone that I feel is important, but this is not the case here)
Re:Squash your commits by Piskvorrr on Saturday April 02, 2016 @03:56AM (#11410686)
Sometimes, having a snapshot is useful even if it's not a snapshot of a completely functional system. "Hmm, this doesn't seem to be a workable approach here; let's commit, checkout an earlier version and branch off that." It's not necessary to push all these experiments (or even keep them in the end), but it might be useful to have them, at least for the moment.
Re:Squash your commits by ghostwriter on Saturday April 02, 2016 @04:15AM (#11410717)
That's what "git stash" is for.
Re:Squash your commits by Piskvorrr on Monday April 04, 2016 @03:33AM (#11420049)
Stash is supported as a first-class citizen by exactly zero tools that I've seen - even though a stash is a commit in all but name ;)