News for hackers. Stuff that matters.
Re:Oxen.ai: Very Fast Unstructured Data Version Control by gschoeni on Monday February 20, 2023 @10:43PM (#34876209)
Creator of the project here, would love to look at what an LFS style integration with git hooks could look like. Let me know if you have any ideas there!
Re:Oxen.ai: Very Fast Unstructured Data Version Control by Zee2 on Tuesday February 21, 2023 @01:37AM (#34877526)
Honestly, I'd just trawl the interwebs for all the reasons folks have had grief with LFS, and then see if Oxen can even do a fraction better. If it can, you'll have many billion-dollar companies that would be highly interested in your solution :)

A few starting points from personal experience would be

- Ensuring git-aware code editors don't try and mangle blob histories when they encounter them - Modern, high-performance blob downloads/version-resolution - Human-readable/debuggable meta-files, where it's easy to recover from bad states (instead of an opaque empty file with a GUID...?)

Re:Oxen.ai: Very Fast Unstructured Data Version Control by yarg on Tuesday February 21, 2023 @07:21AM (#34880276)
If you want blob support, you'd probably need to introduce format specific understanding of file structures.

You'd also need for the blob encoder to be exceptionally stable - if minor changes to the input lead to significant changes to output, you're going to have a lot of trouble.

Something like Google's Courgette (binary patch generation) could help to simplify things, but an old thread mentioned a patent suit.

https://news.ycombinator.com/item?id=2576878

Re:Oxen.ai: Very Fast Unstructured Data Version Control by bfm on Tuesday February 21, 2023 @05:16PM (#34888409)
Shameless plug. I open-sourced https://snapdir.org/ to explore how to solve these exact issues.

It's an early prototype to gather feedback around the ergonomics of the interface and the manifest format, and I would love to learn if people might find it useful enough for the use cases where LFS falls short.