For example: I check out a repository, and create a local feature branch. I create a commit containing the tests for the new feature, then one for the first draft of the new feature, then two or three for bugfixes. Each commit is small, and self-contained, but importantly isn't standalone. If someone checked out the repository in the middle of my chain of commits, they wouldn't have a working product. Then I upload my change for code review. There's no point in reviewing each of my ~5 commits individually: they only make sense to the reviewer as a combined unit. And there's no point in landing them individually: they only make sense for the overall project history as a combined unit.

In a project with many developers (e.g. 1,000 like the Chromium project), every developer has different local practices. Some keep their work based on HEAD of master via rebase, others via merges. Some do test-driven development, some don't. Making the code review the atomic unit of work, rather than the messy string of local commits, helps the project enforce common etiquette, commit formatting, and readable history.

We only allowed squash commits on master because of what you're describing. That is the level where history "made sense". However, for code review, we wanted to support both styles, because there is an advantage sometimes to seeing the sausage being made. For instance someone will refactor something -- maybe change a method name. Then they apply that refactoring at all the call sites. Very conscientious developers would break this into two commits. We didn't want the first commit on master, but it made sense to review this way, because it was easier on the reviewers: change, effect of change on everything else.

I call this "telling a story" with your commits. There's a lot of value in that style if you have the time to do it.

The other style of commit-by-commit reviewing, where I see all of the work in progress commits, I don't find valuable at all and I _definitely_ don't want to see on master.

Only squash when it removes bug that only ever existed on your machine. Everything else should be recorded in the history. Forensics are important to the long term health of your project and you impoverish yourselves by scrubbing the crime scene.

OK. So would you support an IDE that generated one individual commit per keypress? When I type "hello", that's five individual commits each changing a single letter.

That is the true history of what happened, and it's typically recorded in your IDE's undo log in memory, such that you can step back and forward through it. Are you OK with pushing this commit history to your project? That's the true history of what happened, after all, and it could be forensically useful.

When we commit code to share it with other people, we recognize that only a certain level of detail is relevant to them. The physical sequence of letters that I pressed isn't typically relevant to someone else. Rather, the logical set of changes is what affects them.

If you agree that a single commit per letter of keypress is undesirable then you agree with me in principle. There is a finite level of detail that makes sense to share, practically, with current source control systems, and what we're arguing about is how much to share.

I feel like this is important to point out because people frequently make this argument about the "true history" of source, and all that, while neglecting the fact that the commits they choose to ship are already an arbitrary simplification of the "true history".

That would actually be awesome, if the tools supported it. Imagine how easy it would be to find bugs with a bisect if it can drill down to the actual keystroke that introduced a bug.

Realistically, you'd want to back out a few notches though. Say, every time the dev hit Shift+Ctrl+B (or tabbed over to the browser or whatever signifies "build the project" in the environment in question), so that you get an indication that the current state of things was meant to at least compile and run.

But yeah, that's the value of source control. Being able to dig back in history to the exact spot where a bug was born. I can't understand all why so many people here would want to scrub that away.

Apparently introducing interesting problems when people create a log file one byte at a time, up to several hundred megs. (just 100000000 files for one little project, oops)

Not OP but - conceptually? Yes. I've written file formats which preserve undo/redo history, for example. The caveats:

1) I don't want to inadvertently leak my password. This is an issue with things such as local bash command history buffers as well, and unreviewed autocommits in general. You could say it's too useful - to the wrong people!

2) My tooling needs to be built around a different level of granularity as the default. I'm not OK with single letter commits cluttering up my git log, for example. Having them around to drill down into if I need them? Sure.

Per-letter detail is so granular that even undo/redo systems will often squash history states together. Observing your exact typing Cadence / the extra evidence of initial authorship is niche enough that I'm quite willing to sacrifice that level of detail for the sake of performance, maintainability, or basically any and every other excuse you can think of.

> When we commit code to share it with other people, we recognize that only a certain level of detail is relevant to them. The physical sequence of letters that I pressed isn't typically relevant to someone else. Rather, the logical set of changes is what affects them.

I have never seen a codebase with perfect commits. The ones that always give me trouble wrapping my head around are the squashed "logical set of changes", where the set size is way too damn big. Reverse engineering a saner overview from a series of tiny commits is way easier.

And the physical way something was done does matter at times. I'm going to pay way more attention to "whitespace cleanup" commit done by a human than "whitespace cleanup" commit done by vetted tools, for example. In whitespace significant languages, the former may trigger a full code review. Similar concerns with a lot of refactorings, actually.

> If you agree that a single commit per letter of keypress is undesirable then you agree with me in principle.

Per the above, I can only agree with you in practice :)

> There is a finite level of detail that makes sense to share, practically, with current source control systems, and what we're arguing about is how much to share.

Agreed. But I haven't found a single, solitary codebase, where I'd ever argue "less". I can't even recall a single, solitary commit where I'd have ever argued "less". Commit directly to mainline to fix a single character typo? I'll be annoyed if your changelist description was too terse! I want to see:

  Fix the build:  Fix typo 'baz' -> 'bar'
  Fix documentation typo: 'baz' -> 'bar'

A coworker of mine shared he'd collected stats on who made the most commits/day to test a hypothesis. Due to some outliers, he'd decided against it, but thought I'd find the stats amusing. Out of ~50 people (~15 programmers), I (the most recently hired programmer) topped the chart. Second place? The build server account (thanks to nightly build scripts.) I was indeed amused.

https://cgit.freedesktop.org/libreoffice/core/log/

(Don't look at the OpenOffice.org years, they literally took a whole bunch if development work from SVN branches and then merged them in as a single commit and put in single line descriptions with internal tracking numbers and odd project management codes... utter disaster! And of course the branches are now all lost...)

I like the scope of a lot of those commits, although a few are still chunkier looking than I'd like - take that with a grain of salt, though, as I don't have a good enough feel for the codebase to reasonably estimate how much more they could be chunked up. Pretty much everything has a review link, which is nice. I'd expect more back and forth in the comments, but perhaps that's handled out-of-band.

I have lots of nitpicks with the actual changelist descriptions where I'd want things to improve. "Clean up" could mean just about anything - I must go to diff (and expand the context) to understand e.g. https://cgit.freedesktop.org/libreoffice/core/commit/?id=945... properly. I'd be inclined to instead write:

officeipcthread: Cleanup RequestHandler::Enable: Early bail, remove aDummy (just use aUserInstallPath directly), move declarations.

Now I know scope, and the types of changes (refactoring worth reviewing if looking for breakage, not just ignorable whitespace / comment changes.)

No gripes with the overall style on this one, although interacting with a security component, I'd want multiple reviewers: https://cgit.freedesktop.org/libreoffice/core/commit/?id=2a9... .

I'd lean towards linking a screenshot of at least the new version of UI when the file being modified "isn't human readable" (read: is modified with something other than a text editor, even if I can totally read it) which would apply to e.g. https://cgit.freedesktop.org/libreoffice/core/commit/?id=72c... .

EDIT: Formatting, + rationale RE: changelist description.

Thanks for the info, that's really quite insightful. Appreciate you taking the time to give it!

I must be blind - somehow I only saw one the first time around, but they're clearly both there.

> Thanks for the info, that's really quite insightful. Appreciate you taking the time to give it!

I'll take your word for it ;). And no problem.

And that would be nice to have, a vcs that allows you to fold and unfold commits to different levels of granularity.

Wanna have only a straight line history with all feature branches squashed? There you have it. Just need some of them like that? Sure enough. What if you want every time the file was saved? Not a problem, just configure it to commit automatically.

You can squash into master without losing the history of the code review with git. This gives you the best of both worlds, a more accurate history than the one you propose, and a master that isn't broken.

I've wondered if support for a feature like this could finally resolve the "debate" between merging and rebase. The people who like having the full history (whatever that means) would have it, and the people who care about logical changes would see it by default. The cost would be that the repo history would indeed have the full level of detail, but you could imagine supporting a cleanup operation that erases this "origin" information for commits that are sufficiently old, or else simply doesn't pull the info by default and fetches it only when necessary.

(You can kind of simulate this using merge commits, by only looking at one parent, but it doesn't really work out well in practice.)

FWIW, the OpenOffice.org guys had this insane workflow where they squashed all commits into one commit and then out a summary of all the changes into the commit message. Then they scrubbed the private branches.

That methodology is an extreme example of where squashing does NOT make sense.

This is otherwise known as a merge. A single atomic change to master, referencing only the final files, with the first parent referencing the previous master that was also "not" broken.

It has only one change: A second parent, referencing the more accurate history.

If we start without M, and start with master=R2, temporary-feature-branch=R5, I expect the following outcomes:

"squash" will leave us with master=M, and the graph R1 -> R2 -> M. Once temporary-feature-branch is deleted, R3-R5 are forever lost.

"merge" will leave us with master=M, and the graph R1 -> R2 -> M, + R2 -> R3 -> R4 -> R5 -> M

"rebase" will leave us with master=R5, and the graph R1 -> R2 -> R3 -> R4 -> R5. Once temporary-feature-branch is deleted,

The first case loses the history of the code, and thus can't be what we're talking about.

If you call the second case "squash" I find that very confusing. It's also not what github is referring to when talking about "squash your commits" in the parent article, they're talking about the first case. Now, git confuses the issue a bit too - perhaps I should be labeling the second case a "non-fast-forward merge", but I'd still find that clearer than "squash without losing the history."

I had to find when a bug was introduced, and guess what? it was inside a single massive commit touching hundreds of files. I prefer atomic commits with good messages, thanks.

[0] https://git-scm.com/docs/git-bisect

You can upload multiple different versions of a single code review, and reviewers can diff both against the base and against previous versions of the review. This is helpful for showing "stories", responses to comments, and for "my original commit got reverted, so here I've reuploaded it, and then also uploaded the fix, so you can clearly see what's different this time".

This is the style I use for code reviews.

There's not a lot of tooling to support it, at least with GitLab, but git-playback [1] is kind of interesting.

[1]: http://codingfearlessly.com/2012/05/14/git-playback/

There also sounds like a need for `git bisect` to be smarter about DAG traversal, in that it seems to treat the branch as if it were a flattened linear list rather than taking advantage of the merge structure of a branch. This also sounds like something that should be relatively easy to make smarter.

I am really curious how you handled these sort of situations at that scale!

If that were true, the optimal solution is PRs with individual commits that all pass testing. I find it much easier to review a series of small changes for logical correctness than mashing them together into a single PR. Github recently added this as a feature, so I'm not in a completely invisible minority there.

And then, when the review is over, having discreet commits makes git bisecting down to the commit that broke the system more granular.

Unfortunately, that's not the 90% case that I see. Most of the time a multi-commit PR contains N-1 commits of incremental development and one final one that fixes all the tests and typos and removes debugging print statements. Neither the project nor the author benefit from having those intermediate commits integrated verbatim.

I know, it might be too much to ask of people... oh well.

Just because you smash away at code for hours doesn't mean that's how your commit history should look. You can revise history in ways that are beneficial instead of destructive.

Sure they benefit. You can see the thought process that went into the commit. If there's something that seems weird or out of place you can see how it evolved into existence. That can be exceptionally useful.

This is too myopic and explains how you can end up with good code, but bad architecture. A good review ensures both.

I like my PRs to be about high level goals, and I want them made of lots of commits I can review. The commits themselves can be the result of re-basing (which is fine within feature branches) and maybe not truly chronological.

I agree that code reviews should be important, and key to understanding the history of a project at a more useful timescale. I just disagree that they should be "atomic" and that a reviewer (even, or especially, a later code archeologist) may not have reason to inspect or dive into smaller units within code reviews.

Where I think that we may agree is that I feel that even if they shouldn't necessarily be "atomic", I agree that code reviews should probably be first-class objects when talking about and dealing with source control. In git, you can use --no-ff merges today as a useful approximation of code review boundaries (especially with PRs and GitHub's default --no-ff and including linking PR #s). It might be nice to see code reviews or other aggregates of commits/commit graphs be truly first-class citizens of git in some manner.

People who review code for inclusion in a project, want to track meta-progress on issues, want to pin versions for release, etc., mutable history means they can squash fixups or fix your whitespace for you, rebase changesets onto other changesets, have history that reflects the project management strategy, etc.

...small, harmless lies.

Interesting choice of words.

Here's another way to think about squashing private commits for public consumption: programmers do not install keyloggers and upload their entire keystroke history including every Backspace and Ctrl+Z used in their text editor to the repositories. And, most of us wouldn't care about seeing it.

Whether John typed "x = 218^H^H73" or "x = 273" is a meaningless distinction and irrelevant noise. Those spurious ^H Backspaces are equivalent to the twitchy multiple commits in private branches. We really don't want to see them. Think of private noisy commits as an extended workspace of a text editor. If squashing those commits is a lie, the Backspace key without an audited keystroke log is also a lie.

Side note: The other comment downthread about keeping all private commit history for "git bisect" is a red herring. Sometimes a commit will deliberately have broken syntax -- e.g. make a quick commit before getting up to grab a soda -- it won't be CI test worthy. Besides, an automated CI server's cpu cycles can point to an upstream integration/test/qa branch instead of a programmer's private branch.

In a world with infinite storage space and a good UX on top of it, I could absolutely see a case where it might be amazing to have a source control integration with the full undo stack of my editors. VCR roll through someone's efforts Twitch style and grab a box of popcorn as you drinking game your way through their typos...

That said, I definitely will rebase/squash local WIP stuff on local-only branches on my own machine, as I see fit. Yeah, I see those as harmless lies because I really didn't build it that way, but sometimes that's what makes me feel better about publishing that work.

I appreciate you trying to push this conversation towards it's extreme, absurd ends, but I also realize that there are a lot of aesthetic judgments here and I for one lean towards keeping more of the little pieces and the interesting digressions like here's where I totally "brb grabbing a soda" the whole branch and sometimes you taking that break means easier commits for me to review when I'm reviewing your code (whether a code review in a PR immediately, or a research effort down the line) as maybe I need a review break there too. I appreciate not everyone feels the same on this topic.

I didn't think of my example as absurd hyperbole. People actually do use "git commit" on their local unpublished branch as another form of Backspace/Ctrl+Z/Ctrl+S. And just like every text-editor Ctrl+S keystroke is not meaningful, every "git commit" is not meaningful either. A lot of commits are just the programmer's personal unhygienic work-in-progress scratchpad stuff. It's not CI or "git bisect" worthy.

>I for one lean towards keeping more of the little pieces and the interesting digressions like here's where I totally "brb grabbing a soda" the whole branch

I won't dispute that you like whatever you like (VCR-playback of every keystroke mistake) because we're all different. However, I'm pretty sure most busy programmers reviewing pull requests will not appreciate having to wade through all the commits about "cleaned up whitespace" and "savepoint before soda run."

A commit to git is a named snapshot of a file tree. That's it. All the other "worthiness" we ascribe to a commit is communal. A person's unhygienic work-in-progress can give you insight into their thought process or their work ethic. Why did this person give me this particular named snapshot of their file tree? I expect they are tell me (or future them) something and I can respect that artifact and preserve it, just as easily as you might angrily suffer from it and destroy it.

CI or "git bisect" "worthy" are orthogonal considerations and also vary between communities and needs.

If there is something we can agree on here: yes, all commits are not created equal. Even all commits aren't worth preserving equally. That said, I think there are tooling solutions here far beyond "destroy that what you dislike".

Just to be clear, your premise of "give me" is flawed because not every "commit" was meant as a capital "C" Commit-a-Logical-Unit-of-Work. Instead, many commits in private local history are a more mundane commit-as-a-backup-checkpoint-savepoint. The "savepoints" are like "^H Backspace". Asking "why the programmer gave you those particular savepoints" is like asking "why did he gave you those ^H Backspace keystrokes". It's a nonsensical question.

The confusion is that the same "git commit" command is used for 2 very different semantic purposes:

(1) git commit -m "fixed bug #23984" --> as Logical-Unit-Work

(2) git commit -m "wip" --> as meaningless backup/savepoint like Ctrl+S save

The type (2) was for the programmer's internal purposes of safety backups, cleaning up whitespace, typos in comments, reflexive muscle memory of saving often, etc. They have no semantic meaning to "give you". Type (2) commits can have deliberate broken syntax and they're not meant to be built or be bisected.

Type (2) commits should never be discouraged because saving work often (including broken midstream work) is a good habit but from an outsiders perspective of the reviewers upstream, they are way too noisy. The spurious commits could be less than 30 seconds apart with no compile/build step in between.

Linus Torvalds has described his thoughts on rebasing:

http://torvalds-family.blogspot.com.au/2009/06/happiness-is-...

From a strategic standpoint, thinking of that data as anything other than 'destroyed' would be a grave error of judgement. From a retention standpoint the difference between 'could' and 'did' is profound.

That's an issue with GitHub itself, not an issue with the method of squashing commits before merging into master. See gerrit as a prime example of separating reviewing code and your master branch.

I would remind you that you are the one introducing morally charged terms like "lying" when describing rebasing. The fact that you cannot conceded even that those who disagree with you are at least morally good actors is the source of the extremism in this conversation.

I would encourage you to seek hard to understand how it could be that morally good people still conclude that rewriting project commit history is a desirable thing, rather than concluding that they're engaged in self/external deception. At the very least, your conversations about the topic will become more productive.

That's deceptive, wasteful, and has more to do with ego or shallow ideas about "cleanliness" than it does with usefully recording our work processes.

Someone reading the code will not be reading the commit log. If there's gotchas in the code, they should be expressed in comments, not in the commit log.

The commit log, on the other hand, should always be readable for devs, especially newcomers to the project, to get an idea of how to develop small and concise features and how to contribute. Smaller, cleaner and always-buildable commits also make for very easy git bisects.

Git isn't here to record your personal history with the code, it's here to provide developers with an understanding of how the project evolved. It's not deceptive to arrange your commits to make them clean, it's part of creating a clear, understandable, easily-reviewed unit of change.

Very often, you have to piece together what happened and why from the record of what people actually did.

There's no such thing as "personal history" with the code in the context of SCM commits. It's all project history.

This new idea is both deceptive, and honestly, shocking. No SCM system previously has ever been used to actively encourage discarding history, and decades of using these SCM systems ought to have adequately demonstrated why that history matters.

Yes, there absolutely is "personal history" with DVCS.

Your rigid mental model for "SCM commits" is relevant for older tools like cvs/svn with lock-checkout-checkin-unlock procedures. The "commits" in those centralized SCMs are a Really Big Deal.

DVCS is different. When a programmer decides to "fork" a public repository, he copies it to "local" harddrive and it becomes a personal repository. From that fork, the programmer can also create an unpublished branch and that branch is another level of personal scratch space. The commits in this type of environment are not a big deal. With git/dvcs, it shifts the Important Action from the "commit" to the "merge/fast-forward" step. That's why many programmers rebase to optimize how that step looks.

The concept of "personal history" is built-in into the ethos of how Git and DVCS works and it's been a separate concept from "public project history" since day one. The programmer can certainly choose to share every untouched line of personal history but he doesn't have to. Same concept as not sharing a recording of memory buffers and a log of every keystroke from the text editor. Most of us don't care to see any of that.

I'm very aware of what DVCS is. It doesn't require breaking our collaborative processes just because changes are staged locally first.

The keystrokes of ^H Backspace are not a "collaborative process". The idiosyncratic persistence of RAM buffers of a text editor is also not a "collaborative process". Why do you think "git commit" on a local unpublished repo is collaborative? I can only guess that it's because the word "commit" is in "git commit"? Well, syntax is not semantics.

>I'm very aware of what DVCS is.

I think many people with your position know the commands syntax of DVCS but don't actually grok the "draft work" philosophy of DVCS. Therefore you assign label of "collaborative process" to unrelated concepts such as the typing habits the programmer happens to execute on his laptop's harddrive. Unfortunately, the programmer persisting his editor's RAM buffers to disk happens to use the command "git commit" which distorts the thinking and causes people to label that action as "collaborative"!

The "pull request" and "merge" is collaborative. The raw "git commit" is not.

But it's still lying.

The general rules of Linus Torvalds are the only real sane ones in this situation, at least IMO:

1. For a clean commit, make sure you rebase only on your private history. If the branch is public and in any way used by anyone else, then unfortunately it's not something that can be considered "private" any more (at least not unless you give strict warnings that no serious work should be done on the code in the branch until it is finalized).

2. Once your code is committed into a public branch or into the master branch, then you can't change it. No rebasing on this code!

3. Do NOT rebase other people's code, even if it is very messy. If you pull in messy code, then complain about their messy code or try to avoid depending on it. Which basically means that developers that collaborate need to practice pushing and merging clean code.

What I disagree with strenuously, to the point of anger, is having a code as where every line of blame tells me only who made a change and what feature it was for. And I know too many people who think of every feature or pull request as a single commit.

THAT is lying, and I mean that in the judgement-passing definition of the word. You're fucking up the version history intentionally, and you should be stopped.

I'm not in the habit of reading code for fun. I have a job to do, and I want to do it. If I'm looking at the version history it's pretty much only for two reasons. Maybe I know for a fact there was a clever bit of code in this file and it's gone now, so I need to find it. Was it deleted or did you split isn't off into another file? I like those but they don't happen to me very often.

More often I'm reading blame because of a wtf moment. I can't think of a logical reason why the code looks the way it does, but I need to make a change and so I need to know if the person who wrote it was trying to do accomplish something or just confused.

Commit a8bcdef by dingus Implement feature #532

That tells me what changed, who changed it, how it changed and when, but I've lost the why. I know, you probably think "feature 532 is why, duh" but it doesn't tell the story of the intermediate states and why for instance they changed 'users > 1' to 'users >= 1'.

And if there is one class of commit should absolutely and under no circumstances ever ever be squashed it's any commit where you reformatted code. If a line changes and its part of a commit for a feature I'm going to assume that method body was all you, and stop looking. If you 'wrote' it but the commit message is "pretty print" I know I have to go pull a blame on the parent version to keep digging.

However, if you have changed users > 1 to users >= 1 and there is some sort of intrisic logic to it, surely a. you put that reasoning into the commit message, and b. you add a comment into the code briefly explaining the significance?

I don't know about you, but I'd prefer to know about why this is significant within the code, rather than have to dig through the blame logs of a version control system!

It's also extraordinarily cluttered, and it really gets in the way when someone later want to do a 'git bisect' to track down when a bug was introduced.

When I'm working I do frequent little commits just to capture and back up my broken stream-of-thought experiments. None of those are going to be relevant to people who work with this code in the future; how does it benefit them to impose my haphazard process on them?

Of course if there's a way to break up my final commit into more meaningful smaller commits, I do that rather than one monolithic commit.

For example if I clean up some whitespace issues, add some new comments to old code, and implement a new feature, I'll put those in three separate commits even if I originally did the entire change at once. In this case you'll see more commits in the public history than I originally had.

Or if I check in a new version of some external library, add an API call that uses it along with its tests, and add UI code that calls the API, those may be separate commits in that order.

My goal is to make the public history useful to future developers.

I did a rebase once because there was a big mess I was trying to clean up in order to make a merge work. I would be surprised if anyone has ever cared about the details.

(though the counterpoint is that maybe there are parts of the granular history that are just busted for other reasons)

I just don't find it useful to see every commit when someone uses Git like I do, committing all sorts of intermediate broken work while experimenting. Especially not commits that are going to be broken regardless.

In particular, I would kill for support for basically the `git log --first-parent` option in viewing the history of a branch on tools like github and gitlab. Rather than squashing your branch, you make your merge commit have a meaningful commit message (which you do anyways for a squashed commit) so that you don't always have to be viewing the tangled web underneath.

There should be no practical difference between a squashed commit and a merge commit from the perspective of the branch that commit is on (they represent the exact same change from parent to child), but the tooling insists on giving you the most complicated possible view all the time so there is a tangible difference.

In terms of feature branches:

The individual engineer is free to do individual commits in their branch as they need to in order to keep track of their work. Before they submit a pull request, they should rebase and squash all of their commits into a single one that thoroughly describes everything in the feature that is being committed. When used in conjunction with tools like Phabricator, Arcanist, and commit templates, the workflow is very smooth.

When another team member goes to code review their pull request, rather than having to examine multiple individual commits there is only a single one to examine and comment on.

Master history:

Rather than cluttering up the mainline history with 'Did this', 'Did that', 'Merged: Did this', 'Merged: Did that', 'Reverted: Merged: Did this' etc, you get a series of commits that articulately describe what each commit was for. In the event you need to revert a feature because it breaks something, its much easier to revert that single commit than trying to hunt through all of the individual commits from an engineers feature branch. And in that case, if you revert one of the commits from the feature branch it could break something else.

A better approach would be to create multiple small commits that work and are self contained. It's ok for commit N to depend on the preceding commit, but each N should be able to stand on its own.

If devs rebase everything before pushing and push often (therefore also rebasing often), conflicts will happen a lot less often. Devs can also use their private branches for temporarily saving all WIP, squashing/rewording only what makes before submitting the PR or pushing to master.

Anyway, to each their own, and I appreciate your preferences differ from mine.

I can "experientially" state that squash throws away very necessary information for anyone trying to make sense of old code.

Myself I like to do PR and merges with code that can be logically comprehended quickly and doesnt break the build.

a lot of that is due to being emphatic to people that are doing the reviewing. I know they don't know the context as they aren't in the code so I'm showing them a snippet of it, say a new method in s single commit. Then another commit for another method. Finally a third one that actually does some new functionality by combining the two.

I figure that's good enough for people to see that unit tests passed on the two methods and that the system started getting errors after the last commit so to look there for faulty logic.

  Here is a (made up), but generally realistic git log

  git log | grep -i WIP 
  
  mon 5pm - WIP, going to work on this from home 
  tue 4:45pm - WIP, going to work on this from home
  wed 2:30pm - WIP, meeting
  wed 5pm - WIP
  thu Noon - WIP, working from the cafe on my laptop
  fri 5pm - WIP, working from home
  sat 3pm - WIP, heading home sick for the day

Maybe not information that I care to do much more than skim, of course, unless I'm your manager looking for reasons why you might be working from home too much. :) (That said, there's probably some cool deep learning applications here...)

Some of the information, for instance, is that maybe you are working on pieces too large at a time and should find more ways to break them into smaller units of work that you can more easily commit in logical piece at a time rather than "snapshot dumps" between computers.

Like I said, from a hyberbolic standpoint, how the sausage is made isn't pretty and is full of garbage sometimes, but it is informative.

Ideally, every commit that I'm making should be preferably not big, but logically complete and working. The problem is that sometimes I want to work on another branch and I have to commit in the middle of the work so I can checkout into the other branch in which case (without a squashing merge) the "offending" WIP commit would end up in the master branch's history.

  man git stash

Git stash is more dangerous than other git commands, and I've personally witnessed more people losing work with stash accidents than with commit accidents. There may be plumbing commands that can get you there like fsck but the fact of the matter is you are better off committing or branching, from a safety standpoint.

> no rebases, no squashes, sometimes not even fast-forwards

That said, I've been happy to debate the larger implications that there are UI/UX tweaks/story-telling that could make even the work-in-progress non yet published stuff more appealing to publish without needed to mutate it from its as-developed states... But yes, I was mostly speaking to published commits.

I would hate to have done a bisect to land on your commit "WIP, heading home sick for the day" as the one that caused the bug. By all means, create a WIP branch and if you can then push this WIP branch to the main repo, but please use commit squashing into logical units of change when merging into the feature branch or main trunk!

No. That advice is wrong.

Git was designed for those type of low-friction wip commits in private/local/unpublished branches. Many programmers have a concept of "savepoints". Think of "git commit" as an "implementation detail" of that savepoint. Those savepoints were intended to extend the programmer's mental scratch space and never meant for public consumption. Therefore, squash/rebase is a logical followup step to consolidate meaningless savepoints into a meaningful commit.

A private "commit" in git does not have the same semantics as the lock & checkout type of commits in svn/Perforce. Programmers should not be discouraged from making cheap commits as often as they want even with nondescriptive titles because those commits weren't meant for public dissemination anyway. Being judgmental about those wip commits makes no sense.

But it's a lot easier to convince devs that using tools with bad UI elements is hardcore and makes them look smart instead of demanding improvements.

Certainly there are a lot of people that seem prefer imperative mutability, and more power to them, but maybe we learn from all of this and build better tools too.

I agree that code once committed to master should be immutable, but your own private commits should be maleable. It's not a matter of feeling superior or trying to look smarter, it's just really commonsense that you should try to submit easy to understand code changes and remove as much unnecessary extraneous rubbish as possible.

That said, there are certainly ways to better handle the "Oops, syntax error" style of commits than blowing them out after the fact. Some of that is better acknowledging the existing DAG structure of git and realizing that you don't need to 'depth-first search' the commit log every time you look at the commit log. Tools could default to views more like `git log --first-parent` and then let you expand out from there, for instance.

There's also the idea of better drawing lines between the code work and "story telling work" of a commit. Certainly today you could do all your work in progress in a branch and then `git merge --no-ff` it into another branch and there tell the completed story of that commit set after all the "dust has settled" on the stuff that was done in the heat of progress.

It's as much a matter of tooling support and community support (we all make mistakes so we aren't going to dunce cap anyone for making mistakes) as it is a technical problem.

I think we can see that "not all commits are equal" and maybe we want a system for "color these commits as work in progress" followed by "here's the final story for these commits, color it as presentable and ready for code review". I think it's an interesting discussion to continue to have and an opportunity to really think what we want in our source control tools and how they can better help us tell the stories of the code we are working on, for our reviewers and even maybe for our code archeologists.

Imagine working on a fast-changing block of code with 20 or so other people concurrently. If all 20 of those peoples have patches submitted to master for review, and all of those patches have multiple commits each, then every time one gets merged, the others will have to rebase to its changes and fix conflicts for _every single commit_ while the fast forward plays out. It's a horrible experience. It prevents desirable code from getting in as contributors drop out due to the browbeating.

Squashing commits can be the difference between tediously fixing something once vs. tediously fixing it 20 times. No one needs to know that you changed your mind about calling that struct "ConfigOpts" before it was ever introduced upstream.

I'm mostly sort of advocating a "rebase none of the things" approach. Fix conflicts only when they happen in a branch (the GitHub PR system very nicely doesn't let you merge branches that conflict with your target branch and with CI information even better it won't let you merge branches that don't build). It's really not a bad experience.

I would probably be much happier not squashing stuff if you were able to bisect cleanly to points between branch merges. I don't think that's even theoretically possible; which means that when you're bisecting it's quite possible to pick up half-baked mid-branch points that you have to recognise for the broken rubbish they are - lots of false positives there occasionally, and makes bisect a lot less useful. On a repo with nice squashed commits, you tend to be able to narrow down to the feature very quickly - of course those commits then tend to be bigger etc., but I find that less of an issue.

Also, what if there was a tool on top of bisect that could better utilize GitHub PR JSON to target the search pattern? That could even save you some time in the case where you already have CI information attached to your PRs...

[1] http://git.661346.n2.nabble.com/RFC-Add-bad-branch-first-opt...

What's truly meaningful IMO is a git log that reads like a product change log

I have a self-congratulating black belt in source code archeology. With the right tools, most of which are on GitHub, even, such as good commit range diffing, smart uses of tags and branches, and knowing how to navigate the DAG from merge commits (more reason to -no-ff) you have a lot of power in your hands.

«What's truly meaningful IMO is a git log that reads like a product change log»

I appreciate that point of view, but I don't share it. A product change log, I feel, is a bit of marketing/PR that needs some time, love, and editing; I find a git log is for catching snapshots of raw progress and more often useful in seeing what your co-developers are up to, as they are working.

Even commits that lack good commit messages provide valuable information in the form of insight into the author's cumulative thinking/process.

I'm an engineer, and I see the commit history as a tool. When I want to know what a block of code is for and why it was written the way it was, the commit history (if it is clean and granular) will tell me a lot about that, and will point me to authors, issues, features, and requirements where I can learn more. I don't care about the process of producing the code, I care about the end result. I get enough exposure to the process when I'm writing my own code.

I've had it put to me that an Architect deals solely with the art of a project and a Scientist deals solely with the science and theory; it's the work of an Engineer to deal in the practical middle where art meets science (meets the real world).

Sometimes it is easy to overlook (or to want to overlook) the little bits of humanity in the machine; the various sorts of creative chaos in the vast ordered systems; the parts of the code that are art.

There's no easy answers to much of this thread, because it is art, it is aesthetics. There's no "right" answer, just "this looks good and pleasing to me and my team" and working to find that practical Engineering border space between the unwavering art of the Architect and the similarly unwavering logic and discipline of the Scientist.

I'll give you another analogy...If I'm composing a song, and you're not particularly trying to learn how to write songs, what is more relevant to you, the end result or the various drafts and early versions that made it to that end result? For the majority of people who aren't interested in learning the mechanics of songwriting, the "journey" is definitely not as interesting as the destination. For nerds like you and I, that's part of the fun! So I try to keep the original intent of my commits, and preserve their messages in a bullet-pointed list format, to show the individual changes that were made in addition to a higher-level overview of the overall change to the project.

TLDR: Project-level changes are not the same as individual changes, and while both should be represented in commit messages, the project-level changes are overwhelmingly more useful in the future. Git is not about code storage, it's about code communication. It's about developers on the same team communicating with both prose and code in tandem.

Multi-parent commits complicate the use of git. When a commit has a single parent, we can pretend that it's a delta: a patch. (Like it fscking should be in a decent version control system based on some sort of patch theory!) When we "show -p" that commit, we get a diff, which is against its one and only parent. Multiple parents also complicate certain situations. They rear their ugly heads and create an ambiguity. For instance, consider git cherry-pick. If a given commit has just one parent, we can pretend that it's a delta and "git cherry-pick" it. If it has multiple parents, the ugly truth is revealed: a git version isn't actually a delta. If you want that change, you need to specify the parent!

The parent of a commit should be the thing that the work was actually based on: the work that the developer took and massaged to create the new baseline representing the commit. When you have multiple parents, only one of the parents actually meets this definition. The others are arbitrary nodes in the system which are just installed as the parents.

Also, Linus endorses cleaning up your WIP commits before pushing.

Often there will be counter-intuitive bits of code that make sense in "git blame" if you see the original, small, commit that they were created as part of. If they're part of a 1000+ line feature bomb, you lose that important context.

Whilst my small commits were squashed, it shows that even well formed smaller commits that are part of a larger change can often be problematic.

Keeping your commit history clean is important. When I'm bisecting, I don't want to see coding errors like typos and syntax errors, they literally get in the way of the bisect. And when I'm reading through a source file, I'd like to each commit to be significant, or at least entirely relevant to a change. Minor syntax error changes, whilst they can still sneak into the master repository, should be few and far between.

Basically, it also encourages unit testing, rechecking your code, continuous integration, and a raft of other good and best practices around coding. And your colleagues will thank you.

Merge squashing is actually a pretty decent way around this - I'm going to use it as my workflow now. I'll make frequently code commits on a seperate branch, then squash down into another branch, then push this.

It's weird. Git was made for something very specific, Linux kernel development. It made a lot of decisions to support that environment. However most of us don't work in that type of environment.

If you have a private repo for your job you're in a very different environment. At my job mutating history is the opposite of what I want. I don't want people mutating history. Ever!

Part of the problem, in my opinion, is that Git encourages tiny commits. I might take it a step further and say that Git mandates tiny commits. Too tiny in my opinion. When that's forced upon you a clean mechanism is required. But I'm not sure it isn't sweeping another problem under the rug.

But I'm weird. I'm a game developer. We all use Perforce. It just works. You can't fuck it up. You can't ruin history. You can't get stuck. Artists and designers can be trained to use it from scratch in 5 minutes. It's so easy to use there aren't tens of thousands of blog posts desperately trying to explain how easy it is use.

To me it seems like a big dump of files like a network mount with locking and some kind of history. But how the hell is one supposed to write software with it?

It has complicated tools for sharing incomplete work. I don't know how you do code reviews but we have a Perl script for that(!).

In other words: if you think p4 is simple and git is not, it's because of your background.

Perhaps your issue here is that P4 is too simple. You check a file in and it's there forever. Sync latest, change things, check in, voila! Once in P4 there is literally nothing you can do to permanently screw things up.

Binary files are locked so only one person can edit at a time. Text files can be merged. New users may need help resolving conflicts. I recommend Araxis Merge.

Code review tools always exist on top of source control. There are different tools that integrate with Git, Mercurial, SVN, Perforce, etc. What did you use at your old job that you were happy with?

I do have at least some good news for you. If your company is running a semi-recent version of the Perforce server you can use Git with it. You can work 100% in Git if that's what floats your boat. https://www.perforce.com/gitswarm

From my point of view, Perforce is designed from completely the wrong standpoint. The "big dump of files" paradigm makes everything strange in software development. In particular, I find it needlessly difficult to share incomplete work that isn't ready to be merged (In Git, you just commit and push, in P4, you use something else, like the shelf for that).

In contrast with the "big dump of files" paradigm, Git is "create patches and share them". The beauty of this is that "sharing" part works without some dedicated, blessed server and changesets are easily shared over e-mail or any other medium.

I must agree with you that the Git UI is quite bad, inconsistent and not easily discoverable (ie. not something for artists). But the concepts of Git are well-defined and elegant and are thoroughly explained in the README in the first commit[0] of Git(!). To mitigate the issues with the bad UI, I've always found the Git manpages to be fairly clear and useful (but not everyone agrees).

What comes to using Git client with Perforce server, I'd use it if our monorepo workflow wasn't completely incompatible with that.

For me, software development has always been about creating and sharing patches, and that's why Git makes sense for me.

[0] https://github.com/git/git/commit/e83c5163316f89bfbde7d9ab23...

I'm fascinated by your use case because for my professional work I've never created or shared a patch. I just submit the new code and that's it. Possibly to some branch. More than likely straight to main. Merging heavy branches sucks. Even in Git.

But I definitely understand where you're coming from here. There's a heavy learning curve moving from SVN to Git. It's just a radically different way of thinking. You have to break your brain and reshape it. Moving from Git to non-Git is much the same.

When I worked on a Unreal Engine 3 game we often talked about doing things the Unreal way. The Unreal way sucked. It was stupid. Objectively wrong even. However it was important to do many things the Unreal way. Because if you did it your way that would only cause pain and suffering down the road. Because if you change a few bits to be your way then eventually they'll try to interact with bits that do things the Unreal way and it's bad times. So you grit your teeth and do it the Unreal way. If you use Git you gotta do it the Git way. If you use Perforce you gotta use it the Perforce way. As painful as it may be.

> I'm fascinated by your use case because for my professional work I've never created or shared a patch.

I do driver development, and often times we're debugging an issue where toggling some hardware (or software) "mode" can make an issue go away or appear. These are the patches we end up making and sharing with coworkers but never merging. We need to be able to do quick and dirty hacks but we can't be putting that kind of stuff anywhere near master branch. I'd say 4 out of 5 patches I write never get merged (and they're usually really short patches, the code itself wasn't of any measurable effort).

Moving from SVN to Git (or darcs/hg in my case) was certainly a big move. But I guess if you were coming from diff/patch/email/shell script background (like kernel before bitkeeper), it was probably very intuitive :)

This has been a fun, educational HN thread. Thanks for sharing!

I know, and we use this sometimes.

It's just not as easy as pushing to a dev branch in Git creating a patch with "git am" and emailing/copy-pasting to a pastebin or whatever.

Thanks for sharing your point of view and making a very civilized discussion from obviously differing opinions :)

https://www.perforce.com/perforce/r14.2/manuals/cmdref/p4_di...

Branching still sucks in P4. But if it improves your work flow at all I figured it's worth mentioning.

Unless someone checks out everything in the depot by accident..

There is not technical reason a source control tool could not offer multiple 'views' into the commit history. A high level linear view which you can zoom into to see the underlying commits and merges. Why do I have to lose the latter to see the former?

(A) how important the commit is and (B) how many lines of code are changed.

Squash makes sense usually when you have have a branch with multiple small commits that affect the same thing. Doesn't make sense to squash 2 commits affecting large parts of the application just because your commit has to be squashed before merging.

Other than that, maybe all of this is an indication of a need for a meta-UX over the change graph to annotate and describe subgraphs in new ways.

If you want to dig into the actual commit, and the dirty truth under the surface - use Gerrit.. you can even use git itself to pull down the truth.

I'm expecting this github feature to be the same, the pull request probably keeps the truth somehow.

I don't have all the answers of what the tooling should be, I just think this is as a good an opportunity to discuss it as any.

GitHub's long-standing --no-ff merges at least are one way of preserving the code review and it's internal changes directly into the git DAG. This mostly works except for tools like git bisect that treat the DAG as if it were a straight line, rather than making use of the fact that the system already supports complicated graphs.

Furthermore, along the questions of why use two tools to navigate the code repository: I sort of wish that things like GitHub PR comments and code annotations made their way somehow into nodes in the git DAG.

The easiest time to catch a bug is when it's hiding in a 10 line diff, especially before you commit it.

It gets progressively more difficult as the scope of the changeset grows.

It looks like it comes down to the style of the dev/team, because I NEVER do any kind of WIP commit.

Err, it doesn't make it look like the "lie of a" straight line, it makes it into a straight line. Whatever the other developers do, I move the project forward one ball of functionality at a time when their changes are useful to mainline.

When you use software, why do you run a "release" instead of whatever happens to be in the dev's directory when they leave for lunch? Don't you feel dishonest getting the version without the bugs?

> Seeing how the sausage was actually made (no rebases, no squashes, sometimes not even fast-forwards) isn't pretty, but it is meaningful

Perhaps if I was going to hire you it'd be interesting to glance at how you work with nobody looking. Do you keep your desk tidy or not?

But it's absolutely irrelevant to the final project and as such, it shouldn't be stored.

> I trust that. It's real and visceral and how software is actually made

You should read Tracy Kidder's _The Soul of a New Machine_, it's a good read about sausage.

But it's not how you should work because you have choice now.

I'd like to think, however, and I think that this is my larger point in this thread, that maybe we could build better storytelling tools that don't delete/mangle/mutate the actual history so that we can see in the same repository both the story and the raw facts.