News for hackers. Stuff that matters.
Re:Appropriate Uses for SQLite by jstimpfle on Monday September 26, 2016 @07:25AM (#12580903)
This is a bad idea. What you want from text logs is the possibility to start anywhere in the file, look for a nearby newline and know it is the start of a record.

This is a very important property. It's also what makes UTF-8 resilient.

Re:Appropriate Uses for SQLite by hawski on Monday September 26, 2016 @07:57AM (#12581072)
Why can't I have both? I am not against text logs, but it is good to have options. When binary format is an option I say that netstrings are probably better option.
Re:Appropriate Uses for SQLite by jstimpfle on Monday September 26, 2016 @09:50AM (#12581772)
For literal binary data you need to store the length information separately.

Netstrings are one option for ephemeral streams. But they are a bad tradeoff for persistent information: you have to read from the beginning of the stream until you find the relevant information. And a single corrupted byte destroys all the information after it.

Better options for persistent data are (pointer+length)s or memory-pools + ranges.