News for hackers. Stuff that matters.
Re:Check out kilobytes, Git commit dataset changes to a 130 ... by wrfopk on Saturday May 13, 2023 @04:42PM (#35932500)
Looks like you've just reinvented GVFS (https://github.com/microsoft/VFSForGit) for a specific use case? Or is this just a partial clone? Or a shallow clone? Or both? It's unclear from the video if this is 130 GB of current state at the branch head or 130 GB of commit history.
Re:Check out kilobytes, Git commit dataset changes to a 130 ... by eliomattia on Saturday May 13, 2023 @05:14PM (#35932802)
It's like GVFS, but for pieces of a file at a time as well: rows, columns, or cells. In git, you can see in the video the same file diff being overwritten, representing a new piece in every commit. A snapshot is recreated by putting those pieces together. Since S3 pointers are used, it is neither partial nor shallow clone strictly, as all the objects and trees are downloaded in the current implementation, but the pointers enable similar delaying behavior, like with DVC.

Sorry if the repo size is unclear, hope this is better: ~180 kB: current state at the branch head, includes pointers to S3 (exact size depends upon packs and indices), plus full history, also with pointers ~890 MB: current state at the branch head, after downloading all files referenced by pointers in the Git history from S3, plus full history, with pointers ~130 GB: commit history, this is what the repo would weigh on DVC or Git LFS

With increasing repo size (even the 2nd, 890 MB state), this enables working on the 1st (kBs) and still commit changes.