Re:Show HN: We scaled Git to support 1 TB repos
by ltbarcly3 on Tuesday December 13, 2022 @03:02PM
(#33974884)
I think you'll find varying levels of maturity in ML ops. Anyway I think we basically agree, if you use something like this you aren't that mature, and if you are mature you would avoid this thing.
Re:Show HN: We scaled Git to support 1 TB repos
by mickkygoel on Saturday December 17, 2022 @09:39PM
(#34034571)
Building mlops solutions for a big tech. Agree, most mature ml teams are not using git for ml data versioning, but in my experience and user research it’s not due to lack of intent. Teams have been forced to move to other ml data tools in absence of scalable git solution, most of which come with a lot of cognitive overhead for ml engineers who don’t want to spend time adopting several custom tools in their ml pipelines.
Mlops people are very aware of tools that are more suited for the job... even too aware in fact. The entire field is full of tools, databases, etc to the point where it's hard make sense of it. So your comment is a bit weird to me