News for hackers. Stuff that matters.
Re:Many small queries are efficient in SQLite by mb7733 on Tuesday February 16, 2021 @05:12AM (#26152591)
I'm genuinely curious what you mean by this. Of course SQLite imposes a file format... That format is a SQLite database
Re:Many small queries are efficient in SQLite by Dylan16807 on Tuesday February 16, 2021 @05:45AM (#26152738)
> Of course SQLite imposes a file format... That format is a SQLite database

Well this is in a context that rejects zip as being a format. Do you do that? If the answer is no then skip the rest of my post and just note that they're talking about a different definition of 'format'.

-

But in that context:

The amount of structure imposed on you by the sqlite database format is not much more than the structure imposed on you by a zip. I think it's fair to rate them similarly as formats. A zip file is basically a key-value store.

"Zip full of csvs", while awful to use, would impose about the same amount of structure as sqlite does: not much. And zip+csv is not much more elaborate than zip on its own.

Re:Many small queries are efficient in SQLite by samatman on Tuesday February 16, 2021 @05:31PM (#26159591)
In SQLite, I can have a table with a row which is `flag INTEGER NOT NULL CHECK flag == 0 or flag == 1 DEFAULT 0`

Surely this is more structure than a ZIP file, which is merely a way of compressing a directory of files into a single entity, can provide?

Sure the INTEGER part doesn't really do anything...

Re:Many small queries are efficient in SQLite by Dylan16807 on Wednesday February 17, 2021 @02:09AM (#26163589)
That's a feature of the database engine, not the file format. You could attach a similar database engine to a zip backend.
Re:Many small queries are efficient in SQLite by samatman on Wednesday February 17, 2021 @02:50AM (#26163795)
This is an asinine objection.
Re:Many small queries are efficient in SQLite by Dylan16807 on Wednesday February 17, 2021 @03:05AM (#26163881)
I have a better one, if you want it.

A configuration like that comes from the program using sqlite. Just adding sqlite into a system doesn't set up any data formatting like that. Sqlite itself gives you a blank canvas. And a blank canvas is not much of a data format.

Re:Many small queries are efficient in SQLite by samatman on Wednesday February 17, 2021 @05:19PM (#26172587)
You're wrong about this.

The SQLite data format includes the schema, in plain ASCII. This self-documenting nature makes it an excellent data format, I've taken advantage of it numerous times in making use of SQLite-based application file formats.

SQLite is put forth as a basis for an application file format, and by definition it must be sufficiently flexible to accommodate any application. But by including the schema, it is self-documenting as to what the structure is, which ZIP isn't and can't be. QED.

Re:Many small queries are efficient in SQLite by millstone on Tuesday February 16, 2021 @06:12AM (#26152872)
I hope to make the modest point that, if you package your data in a SQLite file or a zip file, then you have both packaging and data problems.

MyCoolSQLApp may read and write a SQLite file with its own schema, but it can't handle an arbitrary SQLite file. Likewise MyCoolZipApp can't handle an arbitrary zip file.

Re:Many small queries are efficient in SQLite by mb7733 on Wednesday February 17, 2021 @11:53PM (#26175989)
I'm not clear on the point here. You can make an application that handles arbitrary SQLite files, (for example: the SQLite shell, or a GUI database broswer), or you can make an application that only uses SQLite files with certain content/schema. Does this mean that SQLite does or does not define a file format?

If not, what do you call the specification of how data is stored in a SQLite file besides a 'file format'?