News for hackers. Stuff that matters.

A comment on Why SQLite Is Taking over with Brian Holt and Marco Bambini. Parent comment

Re:Why SQLite Is Taking over with Brian Holt and Marco Bambini by boffinAudio on Wednesday June 12, 2024 @05:52AM (#40656327)
I used SQLite as the file format for a game system, and it was probably one of the most successful technical decisions I've made in 40 years. The fact that the artists and developers and even managers could use standard database tools to lay out changes to the game levels, characters and even physics behaviours, meant I didn't have to write a ton of tooling to allow them access to all of those things - they just opened up their favourite database GUI, added the data to the sqlite database, and shared the single file around.

Nowadays I just can't see any other way to do this, other than by maintaining a well-ordered .zip file or filesystem structure, with strict validation being something that has to be programmed. I'd just prefer to sqlite all the things.

[ Parent | on HN ]
Re:Why SQLite Is Taking over with Brian Holt and Marco Bambini by jeremyjh on Wednesday June 12, 2024 @06:48AM (#40656601)
That sounds like a decent solution, but I'd rather have all that content in text files so there is meaningful version control with features like diffs and merges. Some engines will store the data this way but give you an editor to manage it, and there are dedicated level editors as well but every project is different and it could be the case none of that would have been helpful to you.
[ Parent | on HN ]
Re:Why SQLite Is Taking over with Brian Holt and Marco Bambini by Karellen on Wednesday June 12, 2024 @07:04AM (#40656676)
AFAIK git has pluggable diff and merge engines. I wonder if it would be possible to create a diff/merge git plugin for sqlite files that does an intermediate conversion to .sql dump files, with some kind of serialisation normalisation so that rows are output in a stable order between runs.
[ Parent | on HN ]
Re:Why SQLite Is Taking over with Brian Holt and Marco Bambini by simonw on Wednesday June 12, 2024 @07:29AM (#40656842)
I haven't tried it with git pluggable diff tools yet, but I wrote a tool that lets me convert a SQLite database into a bunch of plain text files so I can manage them in git: https://github.com/simonw/sqlite-diffable
[ Parent | on HN ]
Re:Why SQLite Is Taking over with Brian Holt and Marco Bambini by TeMPOraL on Wednesday June 12, 2024 @07:25AM (#40656809)
It is. It's a bit of a hassle, because the git mechanism that allows it is considered sensitive, so all users of your repo will need to separately configure this if they want to use it.

Bigger problem for me is that none of the code review systems I know of support this. At work, we've been wishing to have something like this in Gerrit, due to multiple SQLite DBs and other binary documents living in the repo, but as far as we can tell, we'd have to modify Gerrit sources directly to make it happen, which IT would frown at and nobody has time for anyway.

[ Parent | on HN ]
Re:Why SQLite Is Taking over with Brian Holt and Marco Bambini by type0 on Wednesday June 12, 2024 @06:15PM (#40663793)
Not git, but I find it's interesting how Fossil implements version control https://www.fossil-scm.org/home/doc/trunk/www/tech_overview....
[ Parent | on HN ]
Re:Why SQLite Is Taking over with Brian Holt and Marco Bambini by robertlagrant on Wednesday June 12, 2024 @07:38AM (#40656909)
We maintain a compromise with Alembic migration files in source control. This does mean people need to do Alembic things to make changes, but if they can do that little bit it's very good, and fully diffable of course.
[ Parent | on HN ]
Re:Why SQLite Is Taking over with Brian Holt and Marco Bambini by boffinAudio on Wednesday June 12, 2024 @07:44AM (#40656942)
We solved this problem by checking in full sql dumps of the content database into source control, in between major changes. Not that we ever had to check and see what went wrong with the content side - using an sqlite database pretty much ensured that things were in the right format before it hit the filesystem assets, and we had practially zero bugs from introducing this methodology to the project.
[ Parent | on HN ]
Re:Why SQLite Is Taking over with Brian Holt and Marco Bambini by out_of_protocol on Wednesday June 12, 2024 @08:52AM (#40657553)
Thousands of text files laying on bare filesystem is rather slow solution, so additional step to convert between text files and sqlite is needed. Never have seen ready to use solutions, only heard of ad-hoc scripts
[ Parent | on HN ]