News for hackers. Stuff that matters.
Re:Figure out who's leaving the company: dump, diff, repeat by GauntletWizard on Friday February 09, 2024 @01:04AM (#39311893)
I've been seriously considering using Git for all sorts of oft-changing-but-rarely-majorly data. Lists of books in my bookshelves.

The other problem is that I sorta want transactional-database features on top of these things. Git does this well. I also want fast indexing on parts. Git does not do this well. I am considering writing a "standard" for the dumping of sqlite to git, so that I can just delegate this out; Any transaction can be expressed as a git commit, and I can run both at once for both the durability and the reasonable indexing; The sqlite database can be re-created and reindexed whenever, and it also sorta works for backups...

Definitely just spinning my wheels, though. We'll see where databases take us next.

Re:Figure out who's leaving the company: dump, diff, repeat by rrrrrrrrrrrryan on Friday February 09, 2024 @03:43AM (#39312760)
> oft-changing-but-rarely-majorly data

I think you're referring to SCDs, and there are plenty of well-defined ways to track these within relational databases:

https://en.wikipedia.org/wiki/Slowly_changing_dimension

Why git?

Re:Figure out who's leaving the company: dump, diff, repeat by jarofgreen on Friday February 09, 2024 @03:15AM (#39312609)
In your git & sqlite setup, I'm not sure which way round you are thinking - which is the SSOT (Single Source Of Truth) and which is the handy cache.

I've been working on a tool that treats the git repo as the SSOT then lets you dump out all kinds of formats for data work including a sqlite DB. I haven't had as much time as I would like on it but it's at https://pypi.org/project/DataTig/

Re:Figure out who's leaving the company: dump, diff, repeat by ElectricalUnion on Friday February 09, 2024 @02:39AM (#39312425)
Would fossil fit this bill?

Somewhat git-compatible, based on sqlite3.

Re:Figure out who's leaving the company: dump, diff, repeat by JonChesterfield on Friday February 09, 2024 @08:03AM (#39314402)
Specifically the fossil repo is a sqlite database.

I have a cron script stashing whatever is currently in an obsidian vault into a fossil repo. There's a fossil addremove command that makes that very easy. Thus distributed backups of said vault.

Re:Figure out who's leaving the company: dump, diff, repeat by Zambyte on Friday February 09, 2024 @12:10PM (#39317287)
What does "somewhat git-compatible" mean? Can it sometimes use existing git repos, or is the mental model close enough to pick up without learning much?
Re:Figure out who's leaving the company: dump, diff, repeat by ElectricalUnion on Friday February 09, 2024 @12:39PM (#39317764)
> is the mental model close enough to pick up without learning much?

I would say git core concepts are pretty similar to fossil concepts, but actual plumbing implementation details are pretty distinct.

The major difference that I remember from a day-to-day "git porcelain" perspective is that rebases and other types of history rewriting are very discouraged.

For a Rosetta stone of somewhat comparable commands: https://fossil-scm.org/home/doc/trunk/www/gitusers.md

> Can it sometimes use existing git repos

You can, but it's kinda a lot of really slow busywork. And you lose some of the not-file-dvcs features of Fossil, but it is possible.

This page explains how: https://www.fossil-scm.org/home/doc/trunk/www/mirrortogithub...

Re:Figure out who's leaving the company: dump, diff, repeat by rofrol on Friday February 09, 2024 @01:28AM (#39312015)
Re:Figure out who's leaving the company: dump, diff, repeat by GauntletWizard on Friday February 09, 2024 @02:40AM (#39312437)
Dolt is really close and yet just doesn't feel right; I don't want to "commit" between transactions, I want every transaction to be a commit.
Re:Figure out who's leaving the company: dump, diff, repeat by timsehn on Friday February 09, 2024 @11:22AM (#39316421)
Re:Figure out who's leaving the company: dump, diff, repeat by GauntletWizard on Friday February 09, 2024 @12:59PM (#39318047)
That's awesome. I will have to try that out.
Re:Figure out who's leaving the company: dump, diff, repeat by GauntletWizard on Saturday February 10, 2024 @10:32PM (#39332447)
If anyone is coming in late, I have discovered https://immudb.io/ , which claims to be a lot of what I'm looking for. Transactional SQL database, indexed, yet immutable data and verifiable chains of transactions.