Re:Debian Git Monorepo
by MenhirMike on Tuesday April 02, 2024 @02:09PM
(#39908934)
Curious, what challenges would git still have to deal with to work with big repos?
500 GB isn't all that much data in 2024, as long as you can resume a broken network connection on fetch (like using HTTP range requests if using http, and when using git LFS), and while 15 million files is a pain in most file systems because of all the overhead, a relatively modern NVMe SSD should be able to cope with it just fine, but I don't know enough about the innermost internals of git on how it manages files. Is there something that's inherently slow/expensive that doesn't scale well with the number of files?
(Basically, I'm trying to find a non-rhetorical answer to the question of "Why not?")
500 GB isn't all that much data in 2024, as long as you can resume a broken network connection on fetch (like using HTTP range requests if using http, and when using git LFS), and while 15 million files is a pain in most file systems because of all the overhead, a relatively modern NVMe SSD should be able to cope with it just fine, but I don't know enough about the innermost internals of git on how it manages files. Is there something that's inherently slow/expensive that doesn't scale well with the number of files?
(Basically, I'm trying to find a non-rhetorical answer to the question of "Why not?")