News for hackers. Stuff that matters.
Re:Show HN: ION, a JSON alternative – Versatile, compact, fa... by efaref on Wednesday February 03, 2016 @12:18PM (#11027759)
Or you could use an actual RFC standard[1] that has numerous high-quality implementations[2].

[1] https://tools.ietf.org/html/rfc7049 [2] http://cbor.io/

Re:Show HN: ION, a JSON alternative – Versatile, compact, fa... by VStack on Wednesday February 03, 2016 @12:27PM (#11027848)
We have already look at CBOR and MessagePack. They miss the ION tables for compact arrays of data. Here is a link to our comparison to other data formats. http://tutorials.jenkov.com/iap/ion-vs-other-formats.html
Re:Show HN: ION, a JSON alternative – Versatile, compact, fa... by efaref on Wednesday February 03, 2016 @12:36PM (#11027919)
I don't really see how ION tables are an improvement over arrays of arrays, e.g.:

    {
      "headers": [ "a", "b", "c" ],
      "rows": [
        [ 1, 2, 3 ],
        [ 4, 5, 6 ],
      ]
    }
Furthermore, ION appears to require you to know the length of your data up front, whereas you could use CBOR unspecified length arrays to stream data from your database without precalculating the table length. It seems like quite a niche format, though. Most data is not truly tabular.

Also, in the table it claims "Yes" under support for "Cyclic references", and yet further down the page:

> ION has support for expressing cyclic references between objects. At this point this support is not 100% finalized.

So surely this should be "Yes(*)"?

Re:Show HN: ION, a JSON alternative – Versatile, compact, fa... by VStack on Wednesday February 03, 2016 @12:50PM (#11028024)
First of all, lots of results sent back from backend services (or databases) are arrays of objects. So no, tables are not a "niche" format - tables are heavily used.

Second, an array of arrays could mean anything. You have not semantics telling whether the arrays are independent or if the first array is an array of columns for the following arrays. ION Tables add that semantic information.

Third, yes, we could encode everything as text or as raw bytes and leave it up to the user to make sense of it. But that is exactly what we are trying to avoid with ION. We want to give devs a decent standard data format to use, that doesn't require a lot of data encoding choices up front. The encoding options have been thought through already, and sensible choices already made which you can just follow.

Fourth, you can nest ION tables inside ION tables, and thus create a more compact representation of an object graph. Using JSON / CBOR you would need a lot of nested arrays inside arrays to emulate that. Possible, but not exactly pretty.

Re:Show HN: ION, a JSON alternative – Versatile, compact, fa... by VStack on Wednesday February 03, 2016 @01:38PM (#11028532)
Yes also, the TLV nature of ION means that you have to buffer the ION data while writing it, in case you don't know the full size of the embedded ahead of time. However, this is really only a problem for very big messages. HTTP has worked like that for a long time already, and HTTP servers have proven capable of scaling pretty well, won't you agree?

Additionally, if a message does not include its length up front, you are just trading faster write time for slower read time. A node receiving dynamically sized data then has the problem of knowing how much data to allocate for the full message, plus the receiver has to inspect the data as it comes in to see where it ends. With an ION message the receiver knows within the first 4-5 bytes (typically) how big that message will be, and can thus copy the following bytes directly into the perfectly allocated memory area, without having to examine them any further. Since data is most often read more times than written (e.g. data written to a file), we felt that making a tradeoff that favors read speed over write speed made sense.

Re:Show HN: ION, a JSON alternative – Versatile, compact, fa... by brianolson on Wednesday February 03, 2016 @02:38PM (#11029082)
If you're going to send structured records and care about knowing the structure: use protobuf. If you're going to be binary JSON: use RFC 7049 CBOR. http://cbor.io/
Re:Show HN: ION, a JSON alternative – Versatile, compact, fa... by krschultz on Wednesday February 03, 2016 @02:16PM (#11028870)
I don't see Apache Avro in your comparison table. Have you looked at that?
Re:Show HN: ION, a JSON alternative – Versatile, compact, fa... by VStack on Wednesday February 03, 2016 @04:00PM (#11029709)
Yes, we have looked at both Apache Avro and Thrift. The ION encoding is similar to these. Avro uses schemas, just like Protobuf, so it is not so easy to route for intermediate nodes. Thrift uses its own IDL schema language too.
Re:Show HN: ION, a JSON alternative – Versatile, compact, fa... by room271 on Wednesday February 03, 2016 @01:06PM (#11028197)
Have you also looked at Transit?

https://github.com/cognitect/transit-format

Re:Show HN: ION, a JSON alternative – Versatile, compact, fa... by VStack on Wednesday February 03, 2016 @02:10PM (#11028810)
Yes, we have looked at Transit recently. Transit encodes using JSON or MessagePack. We believe ION to be a more versatile data format than MessagePack, so Transit could benefit from serializing to ION instead of MessagePack.
Re:Show HN: ION, a JSON alternative – Versatile, compact, fa... by GordonS on Wednesday February 03, 2016 @01:10PM (#11028242)
Obligatory xkcd: https://xkcd.com/927/
Re:Show HN: ION, a JSON alternative – Versatile, compact, fa... by robalfonso on Wednesday February 03, 2016 @12:27PM (#11027846)
It wouldn't be fun if you didn't re-invent your own wheel but this time did it with the features you want instead of the ones that are almost-identical-but-you-didn't-invent!
Re:Show HN: ION, a JSON alternative – Versatile, compact, fa... by VStack on Wednesday February 03, 2016 @05:37PM (#11030431)
We believe that while ION is similar in encoding to other formats, it is different in aspects that make ION a better all-round data and network message format. See our ION vs.Other Data Formats Page too.

http://tutorials.jenkov.com/iap/ion-vs-other-formats.html

Not every innovation is from scratch, or 100% unique. MessagePack was designed as a binary encoding of JSON. CBOR too. ION was designed to be a binary general purpose data format able to mimic both JSON, XML, CSV and raw bytes (files), plus the most commonly used data types.

Of course there will be similarities when we try to tackle the same problems, but we also believe we had added something new and useful to the mix.