I would think something like this would work:
cat input.json | jq -c --stream '. as $in | select(length == 2 and $in[0][0] == "users") | {}|setpath($in[0][1:]; $in[1])' > users.jsonlines
Output would me a file that looks like this: {"userid1":{"name":"user1"}}
{"userid2":{"name":"user2"}}
Re:Why I’m dumping Firebase for Web
by asadlionpk on Wednesday August 09, 2017 @11:10AM
(#14969831)
I think I have tried something similar. I will try this one too. Thanks a lot!
The parser would have to parse the whole thing before being able to split it.
You could to the splitting yourself (it's just plain text) and create multiple files whose contents is just an array in the format:
```
[
{},
...
]
```
Then you can use JSONStream to load each of those files individually and map/reduce on the contents.
Re:Why I’m dumping Firebase for Web
by vinceguidry on Wednesday August 09, 2017 @11:03AM
(#14969741)
You could loop through the file counting braces, storing line numbers. Then split the file along those line numbers. The smaller files might not have the exact formatting you need to run through a parser, but you should be able to manually adjust it then, hopefully.
But jq fails silently after 12-24 hours of processing. I am still researching this in free time.
If I can just get the keys (like "userid1") I can do the rest from firebase itself.