Re:Check out kilobytes, Git commit dataset changes to a 130 ...
by eliomattia on Saturday May 13, 2023 @05:14PM
(#35932802)
It's like GVFS, but for pieces of a file at a time as well: rows, columns, or cells. In git, you can see in the video the same file diff being overwritten, representing a new piece in every commit. A snapshot is recreated by putting those pieces together. Since S3 pointers are used, it is neither partial nor shallow clone strictly, as all the objects and trees are downloaded in the current implementation, but the pointers enable similar delaying behavior, like with DVC.
Sorry if the repo size is unclear, hope this is better:
~180 kB: current state at the branch head, includes pointers to S3 (exact size depends upon packs and indices), plus full history, also with pointers
~890 MB: current state at the branch head, after downloading all files referenced by pointers in the Git history from S3, plus full history, with pointers
~130 GB: commit history, this is what the repo would weigh on DVC or Git LFS
With increasing repo size (even the 2nd, 890 MB state), this enables working on the 1st (kBs) and still commit changes.
Sorry if the repo size is unclear, hope this is better: ~180 kB: current state at the branch head, includes pointers to S3 (exact size depends upon packs and indices), plus full history, also with pointers ~890 MB: current state at the branch head, after downloading all files referenced by pointers in the Git history from S3, plus full history, with pointers ~130 GB: commit history, this is what the repo would weigh on DVC or Git LFS
With increasing repo size (even the 2nd, 890 MB state), this enables working on the 1st (kBs) and still commit changes.