News for hackers. Stuff that matters.
Re:Git’s database internals I: packed object store by cryptonector on Monday August 29, 2022 @11:20AM (#32639334)
This is a good mental model. The internals are the internals. But the model users see is that a repository is a bag of commits (deltas) that organize into a tree, with branches and tags being names for specific commits. With this model in mind anyone can become a Git power user.
Re:Git’s database internals I: packed object store by stu2b50 on Monday August 29, 2022 @11:35AM (#32639495)
From a user point of view, you should not see commits as diffs or deltas, but as a full snapshots of the contents of the repository.

Delta compression is just an implementation detail for storage optimization. Conceptually each commit is an immutable snapshot in its own right.

Re:Git’s database internals I: packed object store by cryptonector on Monday August 29, 2022 @01:17PM (#32640803)
> From a user point of view, you should not see commits as diffs or deltas, but as a full snapshots of the contents of the repository.

That tends to lead to a merge-centric workflow.

The alternative makes it easy to understand rebase. Ergo it's good.

Re:Git’s database internals I: packed object store by stu2b50 on Monday August 29, 2022 @03:57PM (#32642780)
I don't see any particular reason that'd be the case. Even with rebase, the diff model causes some of git's behavior to no longer make sense. For instance, the fact that rebase causes an entirely new series of commits to be made - this is because rebase is just a layer over cherry-picking commits, and that cherry-picking commits calculates the diffs between two commits then applies the diff as its own, new commit.

If commits were diffs, then that process doesn't make any sense. It only makes sense when you think of commits as snapshots, and cherry-pick ing as taking a commit, calculating the diff with the current commit, and then applying that diff to make a brand new commit.

Re:Git’s database internals I: packed object store by cryptonector on Monday August 29, 2022 @06:47PM (#32644534)
Yes, rebase is a series of cherry-picks. Each commit picked is treated as a set of diffs that are applied with conflict resolution, then committed, then off to the next. The mental model of commits as diffs works, obviously.

> If commits were diffs, then that process doesn't make any sense. It only makes sense when you think of commits as snapshots, and cherry-pick ing as taking a commit, calculating the diff with the current commit, and then applying that diff to make a brand new commit.

Sure it does. It works for me. You can use whatever mental model of Git that you works for you.

Re:Git’s database internals I: packed object store by juped on Monday August 29, 2022 @01:45PM (#32641177)
There are two ways commits can be treated in Git. Most of the time, a commit is a specific tree and some metadata. There are, however, some rebase-like operations that use the commit metadata to treat a commit as a diff against one or more parents which it points to. You're wrong in this discussion because it's about storage, but you do need to flip into this dual view of the commit graph to use rebase-like operations correctly.
Re:Git’s database internals I: packed object store by cryptonector on Monday August 29, 2022 @02:13PM (#32641502)
Exactly. Thinking of commits as diffs helps think about rebasing.
Re:Git’s database internals I: packed object store by josephg on Monday August 29, 2022 @11:36AM (#32639513)
> But the model users see is that a repository is a bag of commits (deltas)

Commits in your .git directory aren’t stored using deltas from a previous commit. Each different version of every file is stored in full.

Re:Git’s database internals I: packed object store by MaxBarraclough on Monday August 29, 2022 @11:55AM (#32639727)
From https://git-scm.com/book/en/v2/Git-Internals-Packfiles :

> You have two nearly identical 22K objects on your disk (each compressed to approximately 7K). Wouldn’t it be nice if Git could store one of them in full but then the second object only as the delta between it and the first?

> It turns out that it can. The initial format in which Git saves objects on disk is called a “loose” object format. However, occasionally Git packs up several of these objects into a single binary file called a “packfile” in order to save space and be more efficient.

...

> What is also interesting is that the second version of the file is the one that is stored intact, whereas the original version is stored as a delta — this is because you’re most likely to need faster access to the most recent version of the file.

Re:Git’s database internals I: packed object store by masklinn on Monday August 29, 2022 @01:03PM (#32640621)
That is what's covered by the article, but as various commenters have noted it's not part of git's model, it's an optimisation of git's storage.

So thinking in those terms for modelling / visualising git's behaviour is nonsensical.

Re:Git’s database internals I: packed object store by MaxBarraclough on Monday August 29, 2022 @01:23PM (#32640886)
It's not hard to make correct statements about the git user interface without being misleading about its internals.

josephg said:

> Each different version of every file is stored in full.

which is an untrue statement about how git internally manages its data. It would be fine had they instead said:

> From the user's point of view, it's as if each different version of every file is stored in full. Internally, git may perform compression, including differential compression.

This isn't the first time I've seen this confusion. This 2014 StackOverflow thread [0] does the same, as does an old HN thread. [1] The solution is clear communication.

[0] https://stackoverflow.com/q/8198105/

[1] https://news.ycombinator.com/item?id=25459757

Re:Git’s database internals I: packed object store by cryptonector on Monday August 29, 2022 @01:22PM (#32640869)
Surely you must know what the word "model" means.
Re:Git’s database internals I: packed object store by juped on Monday August 29, 2022 @01:42PM (#32641125)
It doesn't mean "get out of jail free card for cryptonector being wrong", no.

The simplified model that leaves out tricky implementation details is just "every file that's different is a different blob object, stored in full on its own, under the hash of its contents". If you work with this model, you're basically correct in all your reasoning, except that in reality a bit less storage is used because some objects get packed. If you work with the wrong (wrong! not simplified, wrong!) model "every blob object is stored as base-and-deltas", you won't reason correctly about either how the object store presents itself in plumbing commands or how it's stored.

Re:Git’s database internals I: packed object store by cryptonector on Monday August 29, 2022 @02:16PM (#32641542)
> It doesn't mean "get out of jail free card for cryptonector being wrong", no.

I never said that Git commits are diffs. Keep it civil.

> The simplified model that leaves out tricky implementation details [...]

Models do that.

> If you work with this model, you're basically correct in all your reasoning, except that in reality a bit less storage is used because some objects get packed.

That's it? That's the one detail that should stop me thinking correctly based on this model?