News for hackers. Stuff that matters.
Re:Smaller is better – The rise, fall, and rise of flat file... by OJFord on Sunday May 22, 2022 @01:13PM (#31470445)
For me it's about proprietary vs. readable - unless it's obviously necessary for performance I think I'd still prefer 'flat file' to sqlite, but basically I just want something that isn't an incomprehensible blob, that I can use with other tools etc.

Bonus points for some open standard, but some kind of understandable format even if proprietary is miles ahead of proprietary and cryptic.

I wish Fusion360 for example had a sanely git-able on-disk format like:

    line 0xdeadbeef (0,0) - (123,456)
    line 0x01234567
    constraint parallel 0xdeadbeef 0x01234567
Etc., Or something.

(Actually I suppose git - and incremental backups - is the main reason I say I'd still prefer plaintext to db.)

Re:Smaller is better – The rise, fall, and rise of flat file... by andrewstuart2 on Sunday May 22, 2022 @01:57PM (#31470931)
Perhaps it was deliberate, but your last paragraph (plaintext vs db) really just reinforces your point of standards. There's nothing particularly special about plaintext aside from its total ubiquity.

Everything today uses and understands ASCII/UTF-8 and we've used that, plus some handy human conventions like limited character count between newline characters, to build up tools like git to help manage the changes in smaller chunks.

There's nothing preventing us from doing the same for other formats if they become commonly supported or expected. Much like SQLite and some of the tools cropping up.

Re:Smaller is better – The rise, fall, and rise of flat file... by forgotmypw17 on Sunday May 22, 2022 @08:36PM (#31474233)
> There's nothing particularly special about plaintext aside from its total ubiquity.

But ubiquity is extremely special and useful and essential if you want to maintain ownership of your data.

Re:Smaller is better – The rise, fall, and rise of flat file... by chme on Sunday May 22, 2022 @04:41PM (#31472534)
> There's nothing preventing us from doing the same for other formats if they become commonly supported or expected. Much like SQLite and some of the tools cropping up.

True. I mean if it would be common to put source code and text into sqlite files, then most tools would probably also deal with it. However there is a case about simplicity. For instance writing a diff algorithm on plain text files is much simpler and easy to understand, than writing diff algorithms on text or binary structures.

Re:Smaller is better – The rise, fall, and rise of flat file... by chmod775 on Sunday May 22, 2022 @05:18PM (#31472895)
> For instance writing a diff algorithm on plain text files is much simpler and easy to understand, than writing diff algorithms on text or binary structures.

I'd argue the opposite. Coming up with a good, human readable, diff on unstructured text within acceptable runtime is not that easy. LCS is NP-hard and the naive algorithm has unacceptable performance and memory footprint.

However diffing something structured, especially when it has unique keys like in most databases, is really easy. Structure gives you an easy way to compare data systematically and is handy to express differences.

Re:Smaller is better – The rise, fall, and rise of flat file... by CRConrad on Monday May 30, 2022 @04:22AM (#31556702)
> I think I'd still prefer 'flat file' to sqlite, but basically I just want something that isn't an incomprehensible blob, that I can use with other tools etc.

An Sqlite database may be a binary blob, but thanks to the Sqlite software being open source and ubiquitous, it's pretty easily comprehensible. Sure, not quite as accessible as plain text[1], but not far from it.

So when you need something just a little bit more advanced than plain text (i.e. SQL functionality) it seems like a good compromise that doesn't -- pun intended -- too much compromise openness and accessibility.

___

[1]: But, hey, does "plain" text include UTF-8 nowadays or does it still mean ASCII-only?

Re:Smaller is better – The rise, fall, and rise of flat file... by Blackthorn on Sunday May 22, 2022 @03:10PM (#31471628)
> I wish Fusion360 for example had a sanely git-able on-disk format like:

There are two standards for this common to CAD, called IGES and STEP. They are both plaintext, though their gittability is questionable.

Re:Smaller is better – The rise, fall, and rise of flat file... by stjohnswarts on Sunday May 22, 2022 @04:05PM (#31472188)
They are structured text though so I don't think you can just do deltas like like with code. You'd have to treat them like binaries would you not? That's the way we always kept our XML files in git forced to be binaries.
Re:Smaller is better – The rise, fall, and rise of flat file... by chme on Sunday May 22, 2022 @04:45PM (#31472582)
I would really like to see smudge/clean like-filters for git which convert between git objects and work dir files in deal with structured data or binary files more efficiently. So that diffs are minimal and readable even with those formats.
Re:Smaller is better – The rise, fall, and rise of flat file... by zokier on Sunday May 22, 2022 @05:10PM (#31472831)
Code is (generally) structured too?
Re:Smaller is better – The rise, fall, and rise of flat file... by funstuff007 on Sunday May 22, 2022 @08:12PM (#31474100)
> main reason I say I'd still prefer plaintext to db

Is there a non-proprietary somewhat sane way to do version control for spreadsheets (Excel and LibreOffice)?

Re:Smaller is better – The rise, fall, and rise of flat file... by notriddle on Sunday May 22, 2022 @10:01PM (#31474669)
Tell LibreOffice to save the file as Flat XML.
Re:Smaller is better – The rise, fall, and rise of flat file... by tasuki on Sunday May 22, 2022 @02:17PM (#31471129)
> For me it's about proprietary vs. readable - unless it's obviously necessary for performance I think I'd still prefer 'flat file' to sqlite, but basically I just want something that isn't an incomprehensible blob, that I can use with other tools etc.

That is not exactly what proprietary means. Though I feel the same - plain text can be versioned and managed by various tools and that is great!

Re:Smaller is better – The rise, fall, and rise of flat file... by OJFord on Sunday May 22, 2022 @03:52PM (#31472047)
I was not aware I gave a definition for it. I didn't mean it as an antonym for 'readable', if that's what you mean.

Plaintext could be in a proprietary format but that's far better than proprietary compressed database that's a job to work out what even made it (if that wasn't proprietary itself!). Proprietary plaintext might even be better than open standard blobby format, depending on what you want to do with it. (Better for git and backup, worse for opening in competitor tools.)

Re:Smaller is better – The rise, fall, and rise of flat file... by MaxBarraclough on Monday May 23, 2022 @03:23PM (#31483771)
I think their point was that proprietary is not a synonym for bespoke.

If your code is Free and Open Source, then its file-format is presumably not proprietary, whether or not it makes use of an existing standard format (like JSON) or solution (like SQLite).