News for hackers. Stuff that matters.
Re:Oxen.ai: Very Fast Unstructured Data Version Control by Zee2 on Tuesday February 21, 2023 @01:37AM (#34877526)
Honestly, I'd just trawl the interwebs for all the reasons folks have had grief with LFS, and then see if Oxen can even do a fraction better. If it can, you'll have many billion-dollar companies that would be highly interested in your solution :)

A few starting points from personal experience would be

- Ensuring git-aware code editors don't try and mangle blob histories when they encounter them - Modern, high-performance blob downloads/version-resolution - Human-readable/debuggable meta-files, where it's easy to recover from bad states (instead of an opaque empty file with a GUID...?)

Re:Oxen.ai: Very Fast Unstructured Data Version Control by yarg on Tuesday February 21, 2023 @07:21AM (#34880276)
If you want blob support, you'd probably need to introduce format specific understanding of file structures.

You'd also need for the blob encoder to be exceptionally stable - if minor changes to the input lead to significant changes to output, you're going to have a lot of trouble.

Something like Google's Courgette (binary patch generation) could help to simplify things, but an old thread mentioned a patent suit.

https://news.ycombinator.com/item?id=2576878

Re:Oxen.ai: Very Fast Unstructured Data Version Control by bfm on Tuesday February 21, 2023 @05:16PM (#34888409)
Shameless plug. I open-sourced https://snapdir.org/ to explore how to solve these exact issues.

It's an early prototype to gather feedback around the ergonomics of the interface and the manifest format, and I would love to learn if people might find it useful enough for the use cases where LFS falls short.