News for hackers. Stuff that matters.
Re:Internet Object – A JSON alternative data serialization f... by mccanne on Sunday October 24, 2021 @10:44AM (#28978379)
This is a very real problem being addressed here and I am intrigued by all the great comments in this thread.

In the Zed project, we've been thinking about and iterating on a better data model for serialization for a few years, and have concluded that schemas kind of get in the way (e.g., the way Parquet, Avro, and JSON Schema define a schema then have a set of values that adhere to the schema). In Zed, a modern and fine-grained type system allows for a structure that is a superset of both the JSON and the relational models, where a schema is simply a special case of the type system (i.e., a named record type).

If you're interested, you can check out the Zed formats here... https://github.com/brimdata/zed/tree/main/docs/formats

Re:Internet Object – A JSON alternative data serialization f... by mccanne on Sunday October 24, 2021 @10:50AM (#28978422)
Also, if any of you find problems with the Zed spec(s), we'd love to hear about them. "Now" would be a good time to make changes / fix flaws.
Re:Internet Object – A JSON alternative data serialization f... by petre on Sunday October 24, 2021 @10:58AM (#28978507)
I'd like to see more examples and probably data serialized as zed.
Re:Internet Object – A JSON alternative data serialization f... by mccanne on Sunday October 24, 2021 @11:11AM (#28978631)
There are a few examples in the ZSON spec...

https://github.com/brimdata/zed/blob/main/docs/formats/zson....

And you can easily see whatever data you'd like formatted as ZSON using the "zq" CLI tool, but I just made this gist (with some data from the brimdata/zed-sample-data report) so you can have a quick look (the bstring stuff is a little noisy and an artifact of the data source being Zeek)... https://gist.github.com/mccanne/94865d557ca3de8abfd3eb09e8ac...