News for hackers. Stuff that matters.
Re:Why SQLite Does Not Use Git by forrestthewoods on Saturday July 22, 2023 @09:56PM (#36831876)
> and do it well

If only Git did it well!

Ok that’s not fair. Git is pretty okay for the Linux open source project. But it’s pretty mediocre-to-bad for everything else.

The D is DVCS is a waste of effort. Almost all projects are defacto centralized. In fact the D is anti-pattern that makes things like large binary files a still unsolved problem in Git. And no Git LFS doesn’t count.

Source control should be capable for petabyte scale storage and terabyte scale partial clones, imho.

Git feels like an inferior technology that succeeded and now we’re mostly stuck with it. Maybe someday someone will create a proper Perforce competitor and we can all migrate to something that’s actually good.

Re:Why SQLite Does Not Use Git by hprotagonist on Saturday July 22, 2023 @10:01PM (#36831899)
> The D is DVCS is a waste of effort.

until the de facto centralized remote goes offline and you’re stuck with no local history …

Re:Why SQLite Does Not Use Git by kstrauser on Saturday July 22, 2023 @10:08PM (#36831936)
Or on a plane. It’s nice to have 100% of its functionality when you have 0% of your usual connectivity. Branches, commits, merges, all from the local file system? Yes, please.

There’s a reason we moved off centralized VCS en masse.

Re:Why SQLite Does Not Use Git by forrestthewoods on Saturday July 22, 2023 @10:39PM (#36832086)
Sorry, but you’re completely and objectively wrong. This is a huge misnomer.

Distributed and “offline support” are fully orthogonal features.

Distributed means every user has a full copy of the entire repo including full history. This is a radical and unnecessary limitation on the scope and size of source control.

You can have full support for branching, commits, and merges without a fully distributed repo. There are numerous examples of this.

Re:Why SQLite Does Not Use Git by yjftsjthsd-h on Saturday July 22, 2023 @11:37PM (#36832357)
> Distributed means every user has a full copy of the entire repo including full history.

Then git must not be a DVCS; I use shallow clones all the time.

In any event, centralized systems probably can do offline support, at least in theory, but DVCSs more or less include it for free by default, which is still worth something.

Re:Why SQLite Does Not Use Git by kstrauser on Sunday July 23, 2023 @01:03AM (#36832684)
That's an interesting, and wrong, definition of "wrong".

Git, Mercurial, and the like make every full copy of the repo equal. By convention, we often use central repos like GitHub, and commands like "git fetch" and "git push" as fast ways to sync their contents. We don't have to, though.

I can clone a remote repo, then work independently for months. I do this all the time with the Mastodon server I run. I periodically pull in upstream changes, merge them into my working copy, then push those changes out to my worker instances. I might SSH into one of those servers, fix something specific to it, push the changes out to the other local servers so that they're all synced, and repeat for weeks and months on end.

And because each of those repos are equally "official", all branching and merging operations are equally easy on each one of the peer machines. Although it's conceptually possible that it could be just as easy on a VCS that doesn't work with full clones, I've never seen it.

Re:Why SQLite Does Not Use Git by puchatek on Sunday July 23, 2023 @02:54AM (#36833126)
Interesting usecase. The "git bad for everything else" commenter was probably unaware of it. That said, it's not very representative of what git is mostly used for and where there comparison with fossil is least favorable.
Re:Why SQLite Does Not Use Git by genocidicbunny on Sunday July 23, 2023 @03:41AM (#36833348)
It seems you're still largely talking about mostly text files. The parent you're replying to works in video games, where 99% of the data stored in VCS by volume (if it's even stored in VCS) is not text data. It is things like textures, audio files, processed geometry..etc. It is extremely easy to have one single texture take up more disk space than the entire source code for the game.

> I can clone a remote repo, then work independently for months.

If you're working with more than 0 other people in video games, good frakking luck doing this. If you're working on a large enough game, you're also going to need potentially multiple TB of storage just to have a full shallow copy of the repo (with just the latest copies.) I have worked at a studio where a full deep clone of the repo for just one game was well over 100TB. Let's see you load that up on your laptop.

Re:Why SQLite Does Not Use Git by maccard on Sunday July 23, 2023 @06:30AM (#36834076)
Even not in games, moving away for months at a time with no coordination is a recipe for disaster.
Re:Why SQLite Does Not Use Git by kstrauser on Sunday July 23, 2023 @12:04PM (#36836603)
The don’t use Git for it. Emacs is a crappy Photoshop replacement, but it’s great at it was actually built for. Git isn’t great for video game assets, but it’s great at what it was actually built for.

Use the right tool for the job. The parent poster is complaining that their employer picked the wrong tool for their job, then insisting that the tool sucks.

Re:Why SQLite Does Not Use Git by forrestthewoods on Monday July 24, 2023 @12:36AM (#36843152)
You’re missing the point.

Git is optimized for a specific use case. This prohibits other use cases. I think Git could be slightly tweaked to optimize for and enable different use cases. I think far more users would benefit from the new use cases than the old.

I want to use the right tool for the job. I wish Git were changed slightly so that it could be the right tool.

Git is not capable of handling video game assets. It could be. I wish it were. You may may be happy as is. I think you would actually be happier if my use case were supported. Because I think Git is over optimized for an ultra niche case that doesn’t matter to most users. And yes I realize most users are not game developers.

Git is great at what it was built for. Almost all current Git users do have the same requirements as the thing Git was built for. Therefore Git is a sub-optimal tool for almost all users. The world deserves an optimal tool. The hegemony of Git and GitHub makes it difficult for an optimal tool to be made and therefore all users suffer.

Re:Why SQLite Does Not Use Git by ylyn on Sunday July 23, 2023 @01:42AM (#36832825)
You can't really have branches, commits and merges _while offline_ without a distributed repository.

Let me add to that: history and blame too.

Re:Why SQLite Does Not Use Git by lelanthran on Sunday July 23, 2023 @03:56AM (#36833416)
> Or on a plane. It’s nice to have 100% of its functionality when you have 0% of your usual connectivity. Branches, commits, merges, all from the local file system? Yes, please.

You don't need the VCS to be a distributed VCS to have offline capability.

> There’s a reason we moved off centralized VCS en masse.

No, there's a reason we moved to git specifically. We didn't move to the competing DVCS, did we?

Re:Why SQLite Does Not Use Git by LordShredda on Saturday July 22, 2023 @10:06PM (#36831921)
Not an issue, if remote goes down you can always restore from backup
Re:Why SQLite Does Not Use Git by hprotagonist on Saturday July 22, 2023 @10:33PM (#36832061)
having been the guy who did that: i assure you, doing it is An Issue.
Re:Why SQLite Does Not Use Git by KerrAvon on Sunday July 23, 2023 @12:48AM (#36832623)
I was once responsible for doing this with a Perforce server. Only time in my career a VCS has lost data. This was decades ago now and the data loss might have been due to hardware; I cannot definitively blame Perforce, but man was it shitty to deal with at the time. We migrated to Subversion and never had another issue.
Re:Why SQLite Does Not Use Git by hprotagonist on Monday July 24, 2023 @07:23PM (#36855604)
watching inodes fail before my eyes was … not an experience i’d care to repeat
Re:Why SQLite Does Not Use Git by genocidicbunny on Sunday July 23, 2023 @03:32AM (#36833303)
I would love to see a DVCS that can effectively handle repos of the size I've seen. How well does DVCS work when your repo is about 10TB?
Re:Why SQLite Does Not Use Git by hprotagonist on Monday July 24, 2023 @11:34AM (#36849660)
https://github.com/microsoft/VFSForGit

better than it used to, with the caveat that git in particular is not and has never claimed to be good at versioning blobs.

Re:Why SQLite Does Not Use Git by layer8 on Sunday July 23, 2023 @04:21AM (#36833517)
Local history doesn’t imply D. The D basically means that clients and servers are equivalent. You could have a centralized VCS with client-server asymmetry and still have local history.
Re:Why SQLite Does Not Use Git by fritzo on Saturday July 22, 2023 @10:15PM (#36831970)
That D in DVCS is much appreciated when my power goes off, my internet goes down, etc.
Re:Why SQLite Does Not Use Git by forrestthewoods on Saturday July 22, 2023 @10:46PM (#36832122)
Unrelated. Centralized version control systems are capable of offline mode.
Re:Why SQLite Does Not Use Git by eesmith on Sunday July 23, 2023 @01:56AM (#36832887)
You might appreciate then how fossil improves on git. When that happens with fossil you can also still update the issue tracker and wiki, while many (most?) people depends on a centralized system like GitHub or GitLab for the latter.
Re:Why SQLite Does Not Use Git by gorjusborg on Saturday July 22, 2023 @10:33PM (#36832059)
Binaries aren't source code.

It sounds like you want a versioned file system, not source control.

Git is a beautiful solution for its use case.

Re:Why SQLite Does Not Use Git by forrestthewoods on Saturday July 22, 2023 @10:44PM (#36832105)
Git is a “distributed version control system” (DVCS). You’re just adding the term “source code” for no particular reason.

I make video games. Video games contain far more data than mere source code. Almost all game devs use Perforce because Git is insufficient.

Artists and designers desperately need more version control in their lives. They are mostly absent because their programmers use Git which is insufficient. Its a god damn shame how much of our businesses use “stuff_final_final_05.psd” for “version control”.

Re:Why SQLite Does Not Use Git by gorjusborg on Saturday July 22, 2023 @10:51PM (#36832152)
Seems like you are just complaining that something useful to a great many people isn't perfect for you particular use case.

Use something thats's a better fit for your use case.

There's nothing wrong with a hammer, you just want a screwdriver.

Re:Why SQLite Does Not Use Git by genocidicbunny on Saturday July 22, 2023 @11:39PM (#36832361)
Except it seems the world keeps insisting that I use the hammer to bash on the screws.
Re:Why SQLite Does Not Use Git by happymellon on Sunday July 23, 2023 @02:00AM (#36832899)
The world wants you to use git for binary file versioning?

I have never encountered that. Binary files are generally kept in something like SharePoint, where it is SharePoint which is then hated by everyone.

It's interesting that regardless of the tool, everyone seems to hate it. Perhaps if all tools are wrong, you are trying to force a bad usecase?

Re:Why SQLite Does Not Use Git by forrestthewoods on Sunday July 23, 2023 @03:01AM (#36833155)
> you are trying to force a bad usecase

The world has non-text files that require version control support. Version control systems are mostly bad-to-terrible for anything other than source code. Implying that the use case is wrong is borderline offensive.

It’d be nice if there were some artist/designer friendly version control systems. I can teach someone who has never even heard of version control how to use Perforce in 10 minutes. It’s darn near idiot proof and near impossible to shoot your foot off. Git is not easy to use. This is evidenced by the tens of thousands of blogs explaining how easy and simple to use it is. If that were the case there wouldn’t be need for all those posts!

Someday.

Re:Why SQLite Does Not Use Git by LinXitoW on Sunday July 23, 2023 @04:45AM (#36833614)
Text is a universally understood file format. That means it can be easily diffed, compressed, smartly stored (like storing only changes). That's not true of binaries, at least most of them. That's why I'm pessimistic about a good universal VCS ever existing.

I would also love some kind of good (even decent would be enough) VCS for binary/non-text files. I get a stress rash just watching our designers work with "final_final_2023_1_backup_new.psd".

I think Git is relatively simple in relation to all the complicated things you can do with it. If you don't need/use those complicated things, it of course feels like way too much work for too little pay off.

Re:Why SQLite Does Not Use Git by happymellon on Sunday July 23, 2023 @04:47AM (#36833624)
> The world has non-text files that require version control support

Which is fine, but saying that git is rubbish because it doesn't handle binary files as you want it to is a bad usecase.

How version control would handle diffs between PDFs is not the same as photos, which is not the same as video. They have to be content aware for these sorts of things.

Re:Why SQLite Does Not Use Git by genocidicbunny on Sunday July 23, 2023 @05:34AM (#36833803)
Well, to be frank, git doesn't really handle binary files _at all_. I don't really consider treating binary files as opaque blobs of data as 'handling them'. It's more akin to throwing your hands up and saying 'fuck it, we'll treat it like the text stuff and whatever happens, happens.' Yes, over time git has gained some capability for handling binary data as delta patches, but it is so far away from anything even remotely resembling content-awareness.
Re:Why SQLite Does Not Use Git by happymellon on Sunday July 23, 2023 @06:59AM (#36834235)
Because no one is driving a universal content aware diff, as far as I can tell.

If you just want images then gitlab does a good enough job.

https://gitlab.com/gitlab-org/gitlab-test/-/commit/2f63565e7...

How would we represent video diffs? I don't deal with video so I'm not an expert, but there appears to be people complaining without suggesting solutions.

Re:Why SQLite Does Not Use Git by forrestthewoods on Sunday July 23, 2023 @09:03AM (#36834933)
Not all version controlled files can be merged. That’s fine. Git sucks at working with large binary files. Perforce does not. Therefore all game devs use Perforce.

A theoretical Git2 could provide Perforce-tier support for large binary files. But, imho, this would likely require dropping the D in DVCS. I would gladly trade distributed for large binary file support. Without even a moment’s hesitation. Other people wouldn’t. But I wouldn’t call their use case “bad”.

Git is rubbish for my needs. It’s rubbish for a lot of needs. It’s ok at some things. Mercurial is much better at those things though.

Re:Why SQLite Does Not Use Git by olddustytrail on Sunday July 23, 2023 @09:39AM (#36835197)
Then use Perforce? Why are you complaining that something doesn't exist and then literally naming it in the next sentence!
Re:Why SQLite Does Not Use Git by gorjusborg on Sunday July 23, 2023 @03:01PM (#36838536)
Sure, if someone is forcing you to use a tool unfit for the purpose, that's bound to frustrate you.

That's not really a problem with the tool, though. That's a social problem.

Re:Why SQLite Does Not Use Git by forrestthewoods on Sunday July 23, 2023 @02:12AM (#36832958)
Kind of.

I think Git is a very clunky rock that results in people smashing their fingers and is custom designed for a very specific use case (Linux open source) that isn’t actually relevant to 99.99% of projects.

I think Git is a local minimum that people are ignorantly satisfied with. Everyone is using a mishappen rock and the popularity of this rock is inhibiting the creation of a proper hammer.

I could be wrong! But I am unfortunately forced to used Git due to its ubiquity even though it’s an extremely limited tool. Git does not spark my joy.

Mercurial is very similar to Git but incrementally better. In a different world we’d all use MercurialHub. I wouldn’t be fully happy with Mercurial either. But the point is that Git’s ubiquity has to do with ecosystem factors beyond its intrinsic capabilities and design.

Git is a mediocre hammer, at absolute best. I want a great hammer. Users incorrectly think Git is a great hammer which is making it difficult for an actually great hammer to be built.

Almost all HN users will disagree with me. They’re quite happy with Git and think it’s a great hammer. I think they’re all wrong. But maybe it is me who is wrong! :)

Re:Why SQLite Does Not Use Git by ezst on Sunday July 23, 2023 @02:41AM (#36833083)
Curious why you wouldn't be happy with Mercurial :)
Re:Why SQLite Does Not Use Git by gorjusborg on Sunday July 23, 2023 @11:00AM (#36835967)
I think your peception of 99% of use cases is entirely based on your niche, not reality.

There is far more development going on outside of games.

Re:Why SQLite Does Not Use Git by Delk on Monday July 24, 2023 @05:43AM (#36845185)
To me, Git is more like a Swiss army chainsaw.

It's been quite a while since I used other version control systems (such as Mercurial) to any greater extent, so I'll have to rely on my recollection of the feelings and thoughts I had when I did. I also don't really know what the current state of Mercurial is.

My impression of Git in relation to e.g. Mercurial was that Git was harder to get into. Once I did become comfortable and began to kind of understand how git works, though, it seemed to make it possible to do just about anything I wanted to do with source code. In particular, branches, merging, cherry-picking etc. work well and are fast. Doing local experiments before settling for a solution, rebasing your work on latest (already shared) other developments before publishing it to others, etc. are well enabled by those features.

Limited is not how I would describe Git. In fact, it seems rather versatile, as far as managing source code or other similar line-oriented text-based formats go. Grokking it well enough to use that versatility isn't easy, and some of the terminology it uses is just confusing, but once I got past that, it seems to enable lots of workflows that are useful for actual work on source code.

What it of course probably doesn't do that well is handling large blobs. Most software projects don't actually have lots of those, and when there are some, they're often somewhat static. So for most developers, that's usually not a major limitation of the tool. But it almost certainly is to you.

Another thing, of course, is that it's basically just a tool for keeping track of and managing versions and contents of text files. It's not, by itself, a tool for managing higher-level workflows of an entire development project, or for really handling specific formats, or anything else higher level that's not somehow related to managing source code or something similar. That can also be a major limitation. But I don't think that makes it a poor tool; in lots of programming, keeping track of and managing versions and contents of source code in multiple branches is the central use case. Tools for code reviews and other higher level project workflows can then be built separately on top of that.

When you say it's a mediocre hammer, I think you and other people just have different ideas of what a hammer is and what they typically need out of it.

Mercurial probably also allows for all of those things that lots of people like Git for. I honestly don't quite remember what the differences were or how I felt about them when I used both, except that Mercurial seemed easier to handle at first but Git felt possibly more versatile once I got used to it. I can't quite remember what gave me the latter feeling but it's quite possible the choice really made little difference in the end.

For what it's worth, I think BitBucket used to support Mercurial, so we did kind of have something along the lines of GitHub (at the time) for Mercurial.

To continue along the "mediocre hammer" line, is there something specific about Git that makes it seem only mediocre to you if you considered a hammer to be a tool for handling source code alone? How is it limited in that regard?

Re:Why SQLite Does Not Use Git by genocidicbunny on Saturday July 22, 2023 @11:50PM (#36832397)
Amen. I also work[ed] in video games, and this rings very true to me. Git worked well enough for the programmers, but when you're dealing with artists, producers, designers, writers, audio engineers all needing to share work on assets that are also tied to specific versions of source, Git falls apart.

One studio I was at actually had a custom VCS that was a combination of SVN + homebrew asset versioning that worked decently well. By that I mean we all hated the shit out of having to use it, but it worked better than most anything we could get our hands on at the time for a reasonable price.

Re:Why SQLite Does Not Use Git by KerrAvon on Sunday July 23, 2023 @12:59AM (#36832665)
Ah, this is why you have a bad attitude about the “D” part of “DVCS”, right? The workflow for art is “I need to lock that graphics asset so that no one else edits it while I’m editing it,” which you can’t do with a DVCS. Disallowing exclusive edits is generally what most programming teams want; otherwise it’s a daily occurrence to try to get someone to unlock a file they locked for editing and forgot about.
Re:Why SQLite Does Not Use Git by forrestthewoods on Sunday July 23, 2023 @02:50AM (#36833115)
Not really. Locking of binary assets is a separate topic I’ve ignored.

I don’t actually like or dislike the “distributed” part. I don’t care about it. At all. In the slightest. Almost all projects are defacto centralized. If decentralization was free then great, more features for free.

My experience is that distributed is not free. It appears to me to be a fatal flaw that prevents Git from ever being a good tool for large projects with large binary assets, such as games. So I would sacrifice it in a heartbeat IF it gave me the features I do care about.

Re:Why SQLite Does Not Use Git by erik_seaberg on Sunday July 23, 2023 @08:08AM (#36834601)
Locking is just punting on the problem. If a version control system can't merge non-conflicting changes to the same file by different people, that file is unmaintainable and shouldn't have been checked in as if it were source code. Instead it should be generated from a set of files that can be edited normally.
Re:Why SQLite Does Not Use Git by johannes1234321 on Saturday July 22, 2023 @10:46PM (#36832120)
There are features which make life a lot better when you have them: A local search to explore, a way to stash the working state to try something out etc. and once you got those you have mostly a distributed VCS. True the flexibility of multiple remotes and tracking branches most users probably don't need, but it's quite nice for various things, and be it only a simple backup way.

On an aside: I always chuckle on the "distributed" VCS. Back in the days some people refered to CVS as a distributed VCS, comparing to RCS.

Re:Why SQLite Does Not Use Git by KerrAvon on Sunday July 23, 2023 @12:45AM (#36832601)
Counterpoint: Lots of people actually use and like the D aspect. It’s one of the best things about Git!

Local git repos can be shallow or partial clones if you insist on it. You say elsewhere that a VCS should support an offline mode instead, but if that’s not a copy of the repository, how is it in any way equivalent and what exactly is the difference?

I’ve never understood why people like Perforce; I’ve assumed it’s some kind of weird nostalgia for mainframe user interfaces. I understand that it handles some kinds of large assets better than Git, but Perforce is so bad at everything else.

Re:Why SQLite Does Not Use Git by forrestthewoods on Sunday July 23, 2023 @02:38AM (#36833074)
People don’t necessarily “like” Perforce. However it is functional for professional game projects, and other systems like Git and Mercurial are not. Perforce is ripe for disruption.

> if that’s not a copy of the repository, how is it in any way equivalent and what exactly is the difference?

“Distributed” implies the user machine is at least capable of downloading the full repo history. Git defaults to full clones and has various shallow clone features. It also implies support for extremely convoluted graphs and edge cases.

Offline implies a single central hub you connect to and sync with at some later point. I expect the only use case for downloading full history to be for backup purposes. Almost all operations can be performed without having access to every version of every file.

Re:Why SQLite Does Not Use Git by maccard on Sunday July 23, 2023 @06:16AM (#36834002)
I like perforce. It scales significantly better than git, (running git status on a large project, or cloning one takes significantly longer via git than p4). Not having the entire history of a project locally on disk is a pro to me, not a con. P4merge is orders of magnitude better than git's. Atomically incrementing numbers for changelists are superior to sha's if the first thing you do is implement version numbers on top of git. The default workflow is sane and teachable in all of 10 minutes even to non technical people (designers, artists). I like that history is immutable - there are admin tools available if surgery is needed.

P4 isnt perfect, but it's definitely got significant advantages over git in many areas.