News for hackers. Stuff that matters.
Re:Eden by webmaven on Tuesday April 12, 2022 @05:03PM (#31008204)
> These are all features that primarily have value for repos that push the limits on size, like big monorepos (with a huge amount of files) or game development (with big asset files). But get it right, and you could massively cut down the time it takes to check out a branch and build it.

Would this be a good fit for large machine learning data sets?

Every time a new model comes out that touts having been trained on something like a quarter of a billion images, I ask myself "how the heck does someone manage and version a pile like that?"

Re:Eden by zem on Tuesday April 12, 2022 @08:14PM (#31009743)
would you ever need to version individual images? at a high level, you could version the collection as a whole by filtering by timestamp, and moving deleted files to a "deleted" directory, or maintaining filename and timestamps in a database. I'm sure there are lots of corner cases that would come up when you actually tried to build such a system, but I don't think the overall scheme needs to be as conceptually complex as source code version control
Re:Eden by webmaven on Wednesday April 13, 2022 @12:26PM (#31016659)
> would you ever need to version individual images?

Potentially. It would depend on the image format and what metadata is (or can be) included in the file.