News for hackers. Stuff that matters.
Re:Son: A minimal subset of JSON for machine-to-machine comm... by fanf2 on Sunday December 10, 2017 @06:32AM (#15890977)
This spec mostly concentrates on simplifying the syntax of numbers, but it does not do anything about one of the big omissions in JSON: it does not specify the range of integers or the precision of fractions. There are a lot of interoperability problems in that omission.

(The other thing SON does is require unique keys, which is a good improvement that fixes an interop bug that can cause security vulnerabilities.)

Re:Son: A minimal subset of JSON for machine-to-machine comm... by user5994461 on Sunday December 10, 2017 @08:15AM (#15891244)
There are no integers in javascript, numbers are always represented as double.

Most implementations of json respect that. The error people get are because numbers are converted from double to something else that's not equivalent. Precision is around 15 digits and the usual rounding issues are applicable.

Re:Son: A minimal subset of JSON for machine-to-machine comm... by hanbura on Sunday December 10, 2017 @08:37AM (#15891307)
JSON is not javascript, and you can write valid JSON that isn't valid javascript (using unusual line seperators)

No JSON standard restricts the range of numbers (except for universally prohibiting NaN and infinity, making JSON numbers clearly not IEEE doubles). The standards warns of potential interoperability issues with numbers outside the numbers representable as double, but they do allow all numbers representable in Base 10.

Re:Son: A minimal subset of JSON for machine-to-machine comm... by nly on Sunday December 10, 2017 @10:32AM (#15891810)
You're wrong. There isn't a JSON specification anywhere that says numerics are IEEE 754 doubles. The RFC mentions "good interoperability" can be achieved by using doubles, but does not say this MUST or even SHOULD be supported.
Re:Son: A minimal subset of JSON for machine-to-machine comm... by mehrdadn on Sunday December 10, 2017 @06:59AM (#15891059)
> but it does not do anything about one of the big omissions in JSON: it does not specify the range of integers or the precision of fractions

Could you give an example of a storage format that does do this and the problems that doing so avoids?

Re:Son: A minimal subset of JSON for machine-to-machine comm... by majewsky on Sunday December 10, 2017 @07:37AM (#15891144)
I'm not working with binary protocols that often, so I cannot point to an existing protocol. However, if I had to implement a serialization format that doesn't need to be human-readable, I would never use number literals. I would write numbers as one byte whose value means e.g. "the next 4 bytes are an uint32 in little-endian", followed by just these 4 bytes.

Then you also don't have to worry about whitespace, because there is none. The next field can follow directly afterwards, because the type header byte defines how long exactly the current field is. (For variable-length strings, use something like netstrings.)

The only problem that I can see with this is that Son wants to ensure deterministic encoding, but in my example, 0 could be encoded as an integer of any size or signedness. You could add a rule like "encode every integer as the smallest possible type", but that may have practical implications elsewhere.

I wouldn't be too worried about it, seeing how half-baked Son appears to be. For example:

> Object members must be sorted by ascending lexicographic order of their keys.

Yeah right, because there is only one lexicographic order in the world.

Re:Son: A minimal subset of JSON for machine-to-machine comm... by pegasuscollins on Sunday December 10, 2017 @09:39AM (#15891545)
> Yeah right, because there is only one lexicographic order in the world.

The specification tells you all strings (keys) must be UTF8. There is only one lexicographic ordering of UTF8 strings (RFC3629, page 2). This should probably be explicitly mentioned in the linked specification though.

Re:Son: A minimal subset of JSON for machine-to-machine comm... by majewsky on Sunday December 10, 2017 @03:20PM (#15893286)
> There is only one lexicographic ordering of UTF8 strings.

Then why is LC_COLLATE still a thing?

Also, you don't get to second-guess a spec. If it does not specify everything that it needs to, it deserves the "half-baked" label.

Re:Son: A minimal subset of JSON for machine-to-machine comm... by pegasuscollins on Sunday December 10, 2017 @09:52AM (#15891604)
> The only problem that I can see with this is that Son wants to ensure deterministic encoding, but in my example, 0 could be encoded as an integer of any size or signedness.

To be pedantic: Not really. Assuming your example uses twos complement (what else would it use) and you meant "int32" instead of "uint32", negative zero and positive zero would still be the same encoded value. Also since you said "[u]int32", there is only one size the integer can be: 32 bit. So, in your example, the literal value zero has one and only one unique binary representation, which is "0x00000000"

Re:Son: A minimal subset of JSON for machine-to-machine comm... by majewsky on Sunday December 10, 2017 @09:57AM (#15891631)
I implied that a useful serialization format would include integers of multiple size. When your field only needs a uint16, you don't want to pay the extra 6 bytes to encode as, say, uint64.
Re:Son: A minimal subset of JSON for machine-to-machine comm... by pegasuscollins on Sunday December 10, 2017 @10:01AM (#15891653)
You could either stipulate (as you suggested) to always use the smallest width that still fits a number and then prefix with a length byte but that would be wasteful. Ideally something like LEB128 would be used. At any rate, it's not an issue in terms of guaranteeing a bijective mapping.
Re:Son: A minimal subset of JSON for machine-to-machine comm... by majewsky on Monday December 11, 2017 @03:33AM (#15895830)
Nice, I didn't know about LEB128.
Re:Son: A minimal subset of JSON for machine-to-machine comm... by pegasuscollins on Sunday December 10, 2017 @08:57AM (#15891372)
Pretty much every binary format will encode integers using a fixed width or varlen scheme in "base 2".

This generally is done for two major reasons: First of all, such an encoding is significantly easier and cheaper to parse than a base10+ascii (human-readable) encoding.

I encourage you to write a parser that reads a fixed, 32-bit binary number (1) and another parser that reads a JSON-formatted number string into an internal variable in a classic language like Java or C++. You will immediately see the big difference in complexity. Make sure your parser can also deal with a message that contains more than just one number, i.e. the parser should be able to tell at which byte index an encoded number begins and ends. Even if you're using a language or library where this is hidden from you (e.g. by using parseInt or std::stod) the same work still happens behind the scenes.

The other reason is that for most numbers and fixed/varint encoding schemes, the "binary" representation will be much more compact. Storing the number "1000000" in base10+ascii (human readable) takes at least eight bytes. Storing the same number in a fixed 32-bit integer encoding takes four bytes. Using a varint encoding scheme might allow you to get down to three bytes.

(1) Ignoring stuff like byte order and representation of negative numbers; this is usually fixed in the protocol/format specification.

Re:Son: A minimal subset of JSON for machine-to-machine comm... by sundarurfriend on Sunday December 10, 2017 @12:31PM (#15892410)
There's a github issue on this, titled "Son inherits the weakness of JSON by not specifying the numeric value ranges" [1], in case anyone has anything concrete to suggest.

[1] https://github.com/seagreen/Son/issues/3

Re:Son: A minimal subset of JSON for machine-to-machine comm... by emodendroket on Sunday December 10, 2017 @09:13AM (#15891434)
I'd also like to see a built-in date format instead of requiring the parsing of numbers or strings to get them.