News for hackers. Stuff that matters.
Re:JSON with Sqlite by remus on Friday March 01, 2019 @04:35AM (#19279070)
One pattern that I've used when parsing large json responses is to have a table schema like

    frequently|used|data|in|these|cols|full_json_response
So you pull out the fields you need frequently but have the full json so you can dig in to details the response on an adhoc basis. Not very efficient as you can end up storing a big blob of json, but you're not throwing away any data when you parse the response and your schema stays very manageable. Also gives you the flexibility to add new fields to the schema by pulling the data out of the json.
Re:JSON with Sqlite by oefrha on Friday March 01, 2019 @11:24AM (#19282184)
I use this pattern a lot, too. Just make sure not to `SELECT *`, which could be much slower and waste a lot of RAM in some cases (ORMs using e.g. active record pattern are often susceptible to this when used naively).
Re:JSON with Sqlite by tracker1 on Friday March 01, 2019 @12:44PM (#19283358)
I've also done similar where there are differing systems where the extra data is different, but common bits are, well common. For example supporting direct CC, Paypal and Amazon pay. Putting the common parts of the transaction as separate columns, with the full details in a JSON column.
Re:JSON with Sqlite by TremendousJudge on Friday March 01, 2019 @10:27AM (#19281443)
Completely agree, I do this all the time. This is a useful pattern.