gsu2

> All of a sudden you have operations on billions of tiny files that are amplified by AI usage. And object graphs are so fragmented by nature that it's hard to e.g. prefetch a page into memory and ensure that common git operations mainly touch that page's set of objects.

In theory, that's what git's packfiles are for: loose objects get packed together into a file, and if the file is well-arranged then objects that tend to get read together will also end up getting paged-in together (see also: https://www.kernel.org/pub/software/scm/git/docs/technical/p...).

GitHub in particular has spent a lot of time and effort on packing things well (e.g. https://github.blog/open-source/git/scaling-monorepo-mainten...), so I doubt that the "billions of tiny files" problem is at the root of their recent instability.

This is slop; I stopped reading after this line:

> Fastness in software is like great margins in a book — makes you smile without necessarily knowing why.

EDIT: I didn't say _AI_ slop; it's just not well-written. In addition to the word salad quoted above, there's unsubstantiated jumps in logic and opinions that undercut the premise (e.g. "Speed and reliability are often intuited hand-in-hand" being followed later by an example of a "faster, simpler" application having "reliability issues"; or typewriters being "slow in a relative sense" while then praising simplicity of operation, task-focused design, and observability of state over speed); it feels like the author wanted to list out some random complaints but failed to tie them together in a way that felt worth reading.

EDIT 2: having now skimmed the article a few times, I think what the author actually wanted to say is not that software shouldn't be slow, but that it shouldn't be _frustrating_; "slow" is a very common way to frustrate, but not the only one.

Skimming the comments it is a shocker to see people attacking Jordan Lasker (Cremieux) without even some sort of link to more context.

Moore Tool Company -- manufacturer of some of the machines that NIST uses to measure stuff[1] -- published a book called Foundations of Mechanical Accuracy that's pretty good: https://archive.org/details/FoundationsOfMechanicalAccuracy/...

[1]: https://www.nist.gov/news-events/news/2016/09/meet-bob-pmls-...

I might be mistaken, but wouldn't a git signature already be signing trusted things (i.e. the person making the original signature is trusted), making any attack enabled by the input hash function a second preimage attack (i.e. an attacker onky knows the trusted input, not anything private like the signing key)?

Hash collisions mean you can't trust signatures from _untrusted_ sources, but git signatures don't seem to fit that situation.

This seems bad?

- SSH3 is a bad name: this isn't a successor to SSHv2 and will only cause confusion

- The authors don't seem to understand that SSHv2 predates all of their chosen technologies, and provides "robust and time-tested mechanisms" they claim to be adding

- How is "hiding your server behind a secret link" a feature? This is, at best, security through obscurity, which can be layered on any network protocol (e.g. https://en.wikipedia.org/wiki/Port_knocking); this implies that the authors don't have much of a security background...?

- ...Which explains why they think something as complicated as OpenID Connect is a good thing to add to SSH (i.e. https://security.stackexchange.com/questions/148292/why-is-o...)

- The abstract in the linked paper seems to conflate SSHv1 and SSHv2; I couldn't really bring myself to read much past that

In summary: this seems bad.

> First - if git really didn't care about collision resistance, there wouldn't have been a need to switch to SHA1DC as the hash function. They switched because they care enough that they were willing to accept the performance penalty.

Git didn't _need_ to switch to SHA1DC, but they did because the cost was minimal and it's still a good idea to defend against known attacks.

> Second - imagine this scenario: a user creates two commits with the same hash, one with a valid change and the second with a malicious one. The collision could be created by playing around with some data in a binary file - so, this is a collision attack not 2nd pre-image. The user then submits the change to the upstream and gets it approved.

This is a general problem with binary files: they're hard to properly review. Having unreviewable files in a repository (binaries, machine-generated configs, etc.) is already a security problem; hash collisions would just be one (very difficult) way of exploiting that problem.

> The user maintains a mirror of the upstream repo into which they place the malicious commit. Anyone that pulls from this mirror will think they have the same code as the upstream, even if they compare hashes.

Having people pull data from an attacker-controlled source is a security issue, regardless of hash collisions.

> So don't use an untrusted mirror? I guess - but that is something that should be possible with a strong hash. And if git really didn't want you to do that, it would provide for better ways of tracking where objects were actually pulled from.

Git was designed for collaboration between trusted parties; collaboration between untrusted parties (e.g. pulling changes from untrusted sources) is a much harder problem that git doesn't pretend to solve.

> Anyway, collision attacks are real and can impact git. They just aren't as bad as a 2nd pre-image attack.

Collision attacks are real, but they have yet to impact git (beyond adopting SHA1DC, I guess), despite how big of a target popular git repositories are.

> You review the code in the PR, and perhaps even prove it correct. Later on, the malicious actor can do the swapping trick. (Eg by running a mirroring service for your repository.)

Having a copy of code that is reviewable and then searching for a malicious collision is a preimage attack; extending two chosen prefixes (e.g. one "valid" and one "malicious") until they meet at a hash collision is how most practical (?) collision attacks work. The latter scenario produces large junk sections in the results, which should be obvious under even mild scrutiny.

If the reviewer misses the kilobytes of garbage in the middle of a file they're reviewing, then an attacker can just sneak malicious code in directly without requiring a hash collision.

If the project relies on an effectively unreviewable binary file that could hold kilobytes of junk (like some YAML files I've seen...), then that's already breaking the review process without requiring a hash collision.

Ignoring all of that, anybody grabbing code from an untrusted source is already vulnerable to whatever attacks that untrusted source wants to employ, with "exploiting hash collision" being one of the higher-effort attacks that could be mounted.

Essentially, any repository that would be vulnerable to any of the known hash collision attacks (via bad review, untrusted upstream, etc.) would be vulnerable to more mundane, easier attacks against the same weaknesses that do not depend on hash collisions.

I'm not 100% sure I understand your point, but it sounds like you're concerned about signing something using a weak hash function (i.e. where the hash of something is what actually gets signed)?

If that's the case, then my point is pretty simple: yes, SHA-1 is broken for signing untrusted input (due to weak collision resistance), but it is not broken (so far) for signing trusted input (due to strong preimage resistance).

My point earlier was primarily that the contents of a repository are generally trusted (via mechanisms like code review), and signing trusted content still works even with SHA-1.

Note that certificate signing vulnerabilities (which I assume is why TLS was mentioned?) usually rely on a malicious actor presenting one certificate and then presenting a different cert later; they can't arbitrarily fake existing certs from somebody else.

The analogous scenario for git repositories would be to have a malicious actor make a commit (or blob, tree, etc.) that could be swapped out for another. But if you already have malicious actors able to make commits in your repository, then the hash function doesn't matter: they can cause damage in many, many other ways.

Git isn't relying on collision-resistance, it's relying on second-preimage[0] resistance, which is to say: in order to sneak a hash collision in to a git repository, you have to sneak _something else_ that's already trusted (e.g. via code review) into the repository; collisions can't (yet) be generated for arbitrary hashes.

I haven't heard of any second-preimage attacks against MD5, much less SHA-1, so mlindner was correct in asserting that MD5 would be fine (assuming 128 bits are enough). See also the analysis in [1].

More to the point, if you're able to sneak something into a repository in the first place (e.g. a benign file that generates a collision with a malicious file), then you're probably able to sneak in something more directly (e.g. [2]) that won't rely on both getting something in a trusted repository and then cloning from a different, untrusted source.

[0]: https://en.wikipedia.org/wiki/Preimage_attack

[1]: this is getting a bit old, but should still be relevant? https://electriccoin.co/blog/lessons-from-the-history-of-att...

[2]: https://en.wikipedia.org/wiki/IDN_homograph_attack

Launched ten years ago: https://www.dpreview.com/products/nikon/slrs/nikon_d800

Currently available for ~$600: https://www.ebay.com/sch/i.html?&_nkw=nikon+d800&LH_Sold=1&L...

For anybody running into a similar problem: most git hosting services set up a subdomain that will allow SSH traffic over port 443, e.g. ssh.github.com, altssh.bitbucket.org, altssh.gitlab.com, etc.

This question reminded me of Project Wonderful[1], which I used on several tiny projects in an attempt to cover hosting costs: $5 per month. I never had enough traffic to make anything close to that, but it was pretty fun trying.

[1]: https://en.m.wikipedia.org/wiki/Project_Wonderful

Why not force memcached clients to use the binary protocol instead of the ASCII protocol?

The length check would still be necessary, but spaces (and other arbitrary bytes) would no longer break things.