Re:Show HN: We scaled Git to support 1 TB repos
by blobbers on Tuesday December 13, 2022 @11:55AM
(#33971555)
... why do you have 1TB of source code (you don't! mandatory hacker snark)
Is git really supposed to be used for data?
Or is this just a git-like interface for source control on data?
[ Parent
| on HN ]
Re:Show HN: We scaled Git to support 1 TB repos
by stevelacy on Tuesday December 13, 2022 @11:59AM
(#33971652)
Game development, especially Unreal engine, can produce repos in excess of 1TB.
Git LFS is used extensively for binary file support.
Re:Show HN: We scaled Git to support 1 TB repos
by IshKebab on Tuesday December 13, 2022 @12:04PM
(#33971753)
Git is only not "supposed" to be used for data because it doesn't work very well with data by default. Not because that's not a useful and sensible thing to want from a VCS.
Re:Show HN: We scaled Git to support 1 TB repos
by TillE on Tuesday December 13, 2022 @01:31PM
(#33973532)
It's a fundamentally bad idea because of how any DVCS works. You really don't want to be dragging around gigabytes of obsolete data forever.
Something like git-lfs is the appropriate solution. You need a little bit of centralization.
Re:Show HN: We scaled Git to support 1 TB repos
by IshKebab on Tuesday December 13, 2022 @02:09PM
(#33974130)
Because of how Git's current implementation of DVS works. There's nothing fundamental about it. Git already supports partial clones and on-demand checkouts in some ways, it's just not very ergonomic.
All that's really needed is a way to mark individual files as lazily fetched from a remote only when needed. LFS is a hacky substandard way to emulate that behaviour. It should be built in to Git.