News for hackers. Stuff that matters.
Re:Data Version Control by jiangplus on Sunday October 20, 2024 @12:35AM (#41892887)
How does it compare to Oxen?

https://github.com/Oxen-AI/Oxen

Re:Data Version Control by jFriedensreich on Sunday October 20, 2024 @07:17AM (#41894597)
never heard of oxen but it looks like a super interesting alternative. would love to hear from someone who has experience with both.

my first impression: dvc is made to use with git where there are arbitrary folders handled by dvc INSIDE your git repo, where oxen is an alternative for a separate data repo. also oxen has lots of integration with dataframes and tabular, ai training and infernece data that dvc is missing. on the other hand dvc has a full DAG pipeline engine integrated as well as import/ export and pluggable backends.

Re:Data Version Control by gregschoeninger on Sunday October 20, 2024 @11:17AM (#41895851)
Maintainer of Oxen here, we initially built Oxen because DVC was pretty painfully slow to work with, and had a lot of extra bells and whistles that we didn’t need. Under the hood we optimized the merkle tree structure, hashing algorithms, network protocols, etc to make it speedy when it came to large datasets. We have a pretty nice front end at https://oxen.ai for viewing and querying the data as well.

Happy to answer any thoughts or questions!

Re:Data Version Control by bagavi on Sunday October 20, 2024 @12:46PM (#41896651)
Can this be used with GitHub? If yes, I would shift from dvc immediately