News for hackers. Stuff that matters.
Re:A Git story: Not so fun this time by Borg3 on Tuesday July 02, 2024 @04:34AM (#40854665)
You are right. GIT is not DVFS, its DVCS. It was made to track source code, not binary data. If you are putting binary to DVCS, you are doing something wrong.

But, there are industries that need it, like game industry. So they should use tool that allow that. I heard that Plastic-SCM is pretty decent at it. Never used it so cant tell personally.

Replacing GIT is such a stupid idea. There is no ONE tool to handle all cases. Just use right one for your workflows. I, for example, have a need to version binary files. I know GIT handles them badly, but I really like the tool. Solution? I wrote my own simple DVFS tool for that usecase: dot.exe (138KB)

Its very simple DVFS for personal use, peer to peer syncing (local, TCP, SSH). Data and Metadata are SHA-1 checksumed. Its pretty speedy for my needs :) After weeks of use use I liked it so much, I added pack storage to handle text files and moved all my notes from SVN to DOT :)

Re:A Git story: Not so fun this time by ozim on Wednesday July 03, 2024 @05:43PM (#40870272)
Second that way of thinking, for me GIT is as good as it gets for versioning text files.

Not handling binary files is not a downside for me because GIT should not be a tool to handle binary files versioning and we should use something else for that.

Re:A Git story: Not so fun this time by fragmede on Thursday July 04, 2024 @12:58AM (#40872415)
What do you use when you have a tiny .png or .jpg that needs to live alongside your source code now?
Re:A Git story: Not so fun this time by ozim on Thursday July 04, 2024 @03:04AM (#40872954)
I can put a binary file in GIT repo especially small ones and ones that don't change - things that people want are "handling binary files well", whatever that means, but putting big binaries in GIT or a lot of binary files or versioning them is not the use case for GIT.
Re:A Git story: Not so fun this time by kreetx on Friday July 05, 2024 @11:31AM (#40883608)
I think what they mean by "handling binary files well" is being able to selectively not clone/pull them. Otherwise, binary files are handle as any other files, except you are not shown a diff of them by default.
Re:A Git story: Not so fun this time by Borg3 on Friday July 05, 2024 @02:42PM (#40885035)
You are wrong.. Just start putting files of GBs in size and you will see what happens.
Re:A Git story: Not so fun this time by kreetx on Saturday July 06, 2024 @03:28AM (#40888756)
So I just added a 2.4GB iso and other than taking longer nothing happened. Push and clone also seem to work.
Re:A Git story: Not so fun this time by Borg3 on Saturday July 06, 2024 @05:07AM (#40889168)
Right :) I forgot that everyone is in 64bit systems these days. This change a lot. Anyway, GIT uses single blob for single file content. This is all right, if we speak about source code and text files, as those are not large. Thing changes for arbitrary binary files.
Re:A Git story: Not so fun this time by kreetx on Saturday July 06, 2024 @09:25AM (#40890329)
AFAIK, not much of anything changes (perhaps name some of these changes?).

And what is wrong with a single blob? This seems work great for deduplication, if you were using git to manage a photo library and happen to add something twice.

Re:A Git story: Not so fun this time by Borg3 on Saturday July 06, 2024 @10:28AM (#40890657)
There is several issues depending where you run git on. For example mmap() can be problematic on large files in 32bit OS. On Cygwin it gets even worse.

As for deduplication, I do not think so. If you have single blob, lets say 1GB and you change just 1B, whole blob changes and no more dedup. If you use basic method of static block size, lets say 512KB, this will work much better. Futher, there are more advanced techniques to handle dedups like roling checksums to carve out even smaller sub-block.

Re:A Git story: Not so fun this time by kreetx on Saturday July 06, 2024 @10:49AM (#40890761)
But you're always arguing the odd case: - 32 bit systems - cygwin git on Windows instead of native - expecting deduplication on different files (???)
Re:A Git story: Not so fun this time by Borg3 on Saturday July 06, 2024 @01:43PM (#40891906)
Uh, hold on. Are we trying to discuss objectivily? If not, and you just insist like GIT is best for everything fanboyism, and use it everywhere.. I am not interested.

I pointed out that GIT have some rough cases and its good to be aware of them. I said it already and will say it again: choose right tool for the task.

If your workflow with GIT works, use it. No need to discourage other solutions. Yes, I use ancient platforms and I want DVFS on them too.

Re:A Git story: Not so fun this time by kreetx on Sunday July 07, 2024 @06:47AM (#40896597)
Well, you said something will happen with git and big files, but it seems not much will happen for the majority of users.
Re:A Git story: Not so fun this time by nolist_policy on Thursday July 04, 2024 @02:19AM (#40872747)
Just put it in your git repo.
Re:A Git story: Not so fun this time by nmz on Wednesday July 03, 2024 @10:20PM (#40871830)
DVCS stands for distributed version control system, it has nothing to do with source code?

Maybe you're confusing it with SCM which are source control managers, that's the only ones that handle strict source only, but scm can mean other things.

Re:A Git story: Not so fun this time by Borg3 on Thursday July 04, 2024 @07:49AM (#40874113)
Hard to say.. For me DVCS is more advanced version of DVFS. DVCS can do branching and merging, provides more metadata for revisions etc.. DVFS just do pretty much one thing, store binary blobs. And because binary blobs cannot be easly merged, I would not use it for storage here. But I guess, its just me :)
Re:A Git story: Not so fun this time by nmz on Wednesday July 10, 2024 @11:34PM (#40933460)
What is this dvfs you talk about? desktop virtual file system? Dynamic voltage and frequency scaling? Google doesn't know, or well, the first page doesnt know.

DVCS stands for distributed version control system, it has nothing to do with binary blobs, whether or not it supports it, is an implementation detail.

Re:A Git story: Not so fun this time by Borg3 on Sunday July 14, 2024 @02:18PM (#40962398)
VFS = Versioned File System; D means Distributed.
Re:A Git story: Not so fun this time by pantalaimon on Thursday July 04, 2024 @05:37AM (#40873606)
Isn't there git-annex [0] if you want to store large binary files?

[0] https://git-annex.branchable.com/

Re:A Git story: Not so fun this time by Borg3 on Thursday July 04, 2024 @07:52AM (#40874138)
Yeah I know about git-annex. It might be good solution for big data. In my case, I do NOT want to decouple storage from metadata. I want single repo for single project that is self-contained. Easier to manage, its truly distributed. No need to bother w/ backups because every replica have everything allready. Its good model for several GBs of data.