News for hackers. Stuff that matters.

A comment on Decreasing Gitlab repo backup times from 48 hours to 41 minutes. Parent comment

Re:Decreasing Gitlab repo backup times from 48 hours to 41 m... by 1oooqooq on Friday June 06, 2025 @05:33PM (#44205198)
it could have been longer. I still don't know why they were doing backup bundles with two refs :)
[ Parent | on HN ]
Re:Decreasing Gitlab repo backup times from 48 hours to 41 m... by 6LLvveMx2koXfwn on Friday June 06, 2025 @11:22PM (#44207162)
They weren't, if you look at the fix [1] the dedupe loop was run in all cases, not just those with known dupes, so the performance hit was any bundle with lots of refs.

1.https://github.com/git/git/commit/bb74c0abbc31da35be52999569...

[ Parent | on HN ]
Re:Decreasing Gitlab repo backup times from 48 hours to 41 m... by rjmunro on Saturday June 07, 2025 @02:21PM (#44211490)
But why couldn't they just dedupe the refs from the command line before starting the actual bundling - surely there are never more than a couple of hundred of those (one per branch)?
[ Parent | on HN ]
Re:Decreasing Gitlab repo backup times from 48 hours to 41 m... by 6LLvveMx2koXfwn on Sunday June 08, 2025 @10:14AM (#44217157)
The point is the performance hit had nothing to do with dupe count (which could be zero), and everything to do with ref count.
[ Parent | on HN ]
Re:Decreasing Gitlab repo backup times from 48 hours to 41 m... by nayak on Monday June 09, 2025 @04:28AM (#44222445)
Spot on. Some of our repositories at GitLab can contain millions of references.
[ Parent | on HN ]