But yea, as a Clojure guy sexprs or EDN would be much better.
It's a shame JSON parsers usually default to performance rather than correctness, by using bignums for numbers.
> This specification allows implementations to set limits on the range and precision of numbers accepted
JSON is a terrible interoperability standard.
if it's known and acceptable that LLMs can hallucinate arguments to an API then i don't see how this isn't perfectly acceptable behavior either.
Converting that text to _any_ kind of numerical value is outside the scope of the specification. (At least the JSON.org specification, the RFC tries to say more.)
As a textural format, when you use it for data interchange between different platforms, you should ensure that the endpoints agree on the _interpretation_, otherwise they won't see the same data.
Again outside of the scope of the JSON specification.
Similarly to a "schema-less" DBMS -- you will still have a schema, it will just be in your application code, not enforced by the DBMS.
JSON is a nice balance between convenience and restrictions, but it's still a compromise.
That sentence has four negations and I honestly can't figure out what it means.
(It seems this claim is not true, but at least that’s what the sentence means.)
>That sentence has four negations and I honestly can't figure out what it means.
This example is halfway as bad as the one Orwell gives in my favorite essay, "Politics the the English Language"¹.
Compare and contrast:
>I am not, indeed, sure whether it is not true to say that the Milton who once seemed not unlike a seventeenth-century Shelley had not become, out of an experience ever more bitter in each year, more alien (sic) to the founder of that Jesuit sect which nothing could induce him to tolerate.
Orwell has much to say about either.
_____
¹https://www.orwellfoundation.com/the-orwell-foundation/orwel...
I am not, indeed, sure*,* whether it is not true to say that the Milton *(*who once seemed not unlike a seventeenth-century Shelley*)* had not become *-* out of an experience *-* ever more bitter in each year, more alien (sic) to the founder of that Jesuit sect*,* which nothing could induce him to tolerate.
{X} = JSON parsers technically [are] not following the standard if {reason}
{reason} = [JSON parsers] don't reliably store a number that {what kind of number?}
{what kind of number} = number that is not representable by a IEEE754 double precision float
seems simple
I'd be happy to use s-expressions instead :) Though to GP's point, I suppose we might then end up with JS s-expression parsers that still treat ints and floats interchangeably.
Now, S-expressions as used for programming languages such as Lisp do have numbers, but again Lisp has bignums. As for parsers of Lisp S-expressions written in other languages: if they want to comply with the standard, they need to support bignums.
If types other than string and a list bother you, why don't you stick with those types in JSON?
json.Number is (almost) my “favorite” arbitrary decimal: https://github.com/ncruces/decimal?tab=readme-ov-file#decima...
I'm half joking, but I'm not sure why S-expressions would be better here. There are LISPs that don't do arbitrary precision math.
Yup, and if you’re using JSON in Go you really do need to be using Number exclusively. Anything else will lead to pain.
> I'm half joking, but I'm not sure why S-expressions would be better here. There are LISPs that don't do arbitrary precision math.
Sure, but I’m referring specifically to https://www.ietf.org/archive/id/draft-rivest-sexp-13.html, which only has lists and bytes, and so number are always just strings and it’s up to the program to interpret them.
For RSA-4096, the modulus is 4096 bits = 512 bytes in binary, which (for my test key) is 684 characters in base64 or 1233 characters in decimal. So the base64 version is much smaller.
Base64 is also more efficient to deal with. An RSA implementation will typically work with the numbers in binary form, so for the base64 encoding you just need to convert the bytes, which is a simple O(n) transformation. Converting the number between binary and decimal, on the other hand, is O(n^2) if done naively, or O(some complicated expression bigger than n log n) if done optimally.
Besides computational complexity, there's also implementation complexity. Base conversion is an algorithm that you normally don't have to implement as part of an RSA implementation. You might argue that it's not hard to find some library to do base conversion for you. Some programming languages even have built-in bigint types. But you typically want to avoid using general-purpose bigint implementations for cryptography. You want to stick to cryptographic libraries, which typically aim to make all operations constant-time to avoid timing side channels. Indeed, the apparent ease-of-use of decimal would arguably be a bad thing since it would encourage implementors to just use a standard bigint type to carry the values around.
You could argue that the same concern applies to base64, but it should be relatively safe to use a naive implementation of base64, since it's going to be a straightforward linear scan over the bytes with less room for timing side channels (though not none).
Why you wouldn't just use the hexadecimal that everyone else seems to use I don't know. There seems to be a rather arbitrary cutoff where people prefer base64 to hexadecimal.
Python 3.13.3 (main, May 21 2025, 07:49:52) [GCC 14.2.0] on linux
Type "help", "copyright", "credits" or "license" for more
information.
>>> import json
>>>
json.loads('47234762761726473624762746721647624764380000000000000000000000000000000000000000000')
47234762761726473624762746721647624764380000000000000000000000000000000000000000000 >> import json, decimal
>> j = "47234762761726473624762746721647624764380000000000000000000000000000000000000000000"
>> json.loads(j, parse_float=decimal.Decimal, parse_int=decimal.Decimal)
Decimal('47234762761726473624762746721647624764380000000000000000000000000000000000000000000')
This way you avoid this problem: >> import json
>> j = "0.47234762761726473624762746721647624764380000000000000000000000000000000000000000000"
>> json.loads(j)
0.47234762761726473
And instead can get: >> import json, decimal
>> j = "0.47234762761726473624762746721647624764380000000000000000000000000000000000000000000"
>> json.loads(j, parse_float=decimal.Decimal, parse_int=decimal.Decimal)
Decimal('0.47234762761726473624762746721647624764380000000000000000000000000000000000000000000') >>> s="1"+"0"*4300
>>> json.loads(s)
...
ValueError: Exceeds the limit (4300 digits) for integer string conversion:
value has 4301 digits; use sys.set_int_max_str_digits() to increase the limit
This was done to prevent DoS attacks 3 years ago and have been backported to at least CPython 3.9 as it was considered a CVE.Relevant discussion: https://news.ycombinator.com/item?id=32753235
Your sibling comment suggests using decimal.Decimal which handles parsing >4300 digit numbers (by default).
I feel like not understanding why JSON won out is being intentionally obtuse. JSON can easily be hand written, edited, and read for most data. Canonical S-expressions are not as easy to read and much harder to write by hand; having to prefix every atom with a length makes is very tedious to write by hand. If you have a JSON object you want to hand edit, you can just type... for an Canonical S-expression, you have to count how many characters you are typing/deleting, and then update the prefix.
You might not think the ability to hand generate, read, and edit is important, but I am pretty sure that is a big reason JSON has won in the end.
Oh, and the Ruby JSON parser handles that large number just fine.
So of course, ACME is based around a format whose entire reason d'etre is being written and read by hand.
It's weird.
[1]: although being easy to read by humans is an advantage when debugging why something isn't working.
I didn’t feel like my comment was the right place to shill for an alternative, but rather to complain about JSON. But since you raise it.
> JSON can easily be hand written, edited, and read for most data.
So can canonical S-expressions!
> Canonical S-expressions are not as easy to read and much harder to write by hand; having to prefix every atom with a length makes is very tedious to write by hand.
Which is why the advanced representation exists. I contend that this:
(urn:ietf:params:acme:error:malformed
(detail "Some of the identifiers requested were rejected")
(subproblems ((urn:ietf:params:acme:error:malformed
(detail "Invalid underscore in DNS name \"_example.org\"")
(identifier (dns _example.org)))
(urn:ietf:params:acme:error:rejectedIdentifier
(detail "This CA will not issue for \"example.net\"")
(identifier (dns example.net))))))
is far easier to read than this (the first JSON in RFC 8555): {
"type": "urn:ietf:params:acme:error:malformed",
"detail": "Some of the identifiers requested were rejected",
"subproblems": [
{
"type": "urn:ietf:params:acme:error:malformed",
"detail": "Invalid underscore in DNS name \"_example.org\"",
"identifier": {
"type": "dns",
"value": "_example.org"
}
},
{
"type": "urn:ietf:params:acme:error:rejectedIdentifier",
"detail": "This CA will not issue for \"example.net\"",
"identifier": {
"type": "dns",
"value": "example.net"
}
}
]
}
> for an Canonical S-expression, you have to count how many characters you are typing/deleting, and then update the prefix.As you can see, no you do not.
But, I mean, they're basically isomorphic with like 2 things exchanges ({} and [] instead of (); implicit vs explicit keys/types).
S expressions may well be better. But I don’t think S expressions are better enough to be able to overcome json’s inertia.
(urn:ietf:params:acme:error:malformed
(detail "Some of the identifiers requested were rejected")
(subproblems ((urn:ietf:params:acme:error:malformed
(detail "Invalid underscore in DNS name \"_example.org\"")
(identifier (dns _example.org)))
(urn:ietf:params:acme:error:rejectedIdentifier
(detail "This CA will not issue for \"example.net\"")
(identifier (dns example.net))))))
the alignment of the values makes them easier to pick out and gives a visual structurebut, i would also argue that the two examples are not equivalent. what is explicitly specified as "type" and "value" in the json data, is implied in the s-expression data. either format is fine, but it would be better to compare like for like:
an s-expression equivalent for the json example would look like this:
((type urn:ietf:params:acme:error:malformed)
(detail "Some of the identifiers requested were rejected)
(subproblems
((type urn:ietf:params:acme:error:malformed)
(detail "Invalid underscore in DNS name \"_example.org\"")
(identifier
(type dns)
(value _example.org)))
((type urn:ietf:params:acme:error:rejectedIdentifier)
(detail "This CA will not issue for \"example.net\"")
(identifier
(type dns)
(value example.net)))))
or the reverse, a json equivalent for the s-expression example: {
"urn:ietf:params:acme:error:malformed":
{
"detail": "Some of the identifiers requested were rejected",
"subproblems":
[
"urn:ietf:params:acme:error:malformed":
{
"detail": "Invalid underscore in DNS name \"_example.org\"",
"identifier":
{
"dns": "_example.org"
}
},
"urn:ietf:params:acme:error:rejectedIdentifier"
{
"detail": "This CA will not issue for \"example.net\"",
"identifier":
{
"dns": "example.net"
}
}
]
}
}
a lot of the readability depends on the formatting. we could format the json example more dense: {"urn:ietf:params:acme:error:malformed": {
"detail": "Some of the identifiers requested were rejected",
"subproblems": [
"urn:ietf:params:acme:error:malformed": {
"detail": "Invalid underscore in DNS name \"_example.org\"",
"identifier": {
"dns": "_example.org" }},
"urn:ietf:params:acme:error:rejectedIdentifier": {
"detail": "This CA will not issue for \"example.net\"",
"identifier": {
"dns": "example.net" }}]}}
doing that shows that the main problem that makes json harder to read is the quotes around strings.because if we spread out the s-expression example:
(urn:ietf:params:acme:error:malformed
(detail "Some of the identifiers requested were rejected")
(subproblems
((urn:ietf:params:acme:error:malformed
(detail "Invalid underscore in DNS name \"_example.org\"")
(identifier
(dns _example.org)
)
)
(urn:ietf:params:acme:error:rejectedIdentifier
(detail "This CA will not issue for \"example.net\"")
(identifier
(dns example.net)
)
)
)
)
)
that doesn't add much to the readability. since, again, the primary win in readability comes from removing the quotes.It's not for me. I'd literally take anything over csexps. Like there is nothing that I'd prefer it to. If it's the only format around, then I'll just roll my own.
May I suggest perl regex's? :)
As for s-expressions vs json, there are pros and cons to each. S-expressions don't have any way to encode type information in the data itself, you need a schema to know if a certain value should be treated as a number or a string. And it's subjective which is more readable.
[1]: https://en.m.wikipedia.org/wiki/Canonical_S-expressions
927 can be avoided, but it's way harder than it seems, which is why we have the proliferation of standards that fail to become universal.
I’ve always used ‘canonical S-expressions’ to refer to Rivest’s S-expressions proposal: https://www.ietf.org/archive/id/draft-rivest-sexp-13.html, a proposal which has canonical, basic transport & advanced transport representations which are all equivalent to one another (i.e., every advanced transport representation has a single canonical representation). I don’t know where I first saw it, but perhaps it was intended to distinguish from other S-expressions such as Lisp’s or Scheme’s?
Maybe I should refer to them as ‘Rivest S-expressions’ or ‘SPKI S-expressions’ instead.
> S-expressions don't have any way to encode type information in the data itself, you need a schema to know if a certain value should be treated as a number or a string.
Neither does JSON, as this whole thread indicates. This applies to other data types, too: while a Rivest expression could be
(date [iso8601]2025-05-24T12:37:21Z)
JSON is stuck with: {
"date": "2025-05-24T12:37:21Z"
}
> And it's subjective which is more readable.I really disagree. The whole reason YAML exists is to make JSON more readable. Within limits, the more data one can have in a screenful of text, the better. JSON is so terribly verbose if pretty-printed that it takes up screens and screens of text to represent a small amount of data — and when not pretty-printed, it is close to trying to read a memory trace.
Edit: updated link to the January 2025 proposal.
But that perhaps hints at some reasons that formats like JSON tend to win popularity contests over formats like Rivest's. JSON is a single format for authoring and reading, which doesn't address transport at all. The name is short, pronounceable (vs. "spikky" perhaps?), and clearly refers to one thing - there's no ambiguity about whether you might be talking about a transport encoding instead,
I'm not saying these are good reasons to adopt JSON over SPKI, just that there's a level of ambition in Rivest's proposal which is a poor match for how adoption tends to work in the real world.
There are several mechanism for JSON transport encoding - including plain old gzip, but also more specific formats like MessagePack. There isn't one single standard for it, but as it turns out that really isn't that important.
Arguably there's a kind of violation of separation of concerns happening in a proposal that tries to define all these things at once: "a canonical form ... two transport representations, and ... an advanced format".
But still, I think if the original JSON spec had been longer and more comprehensive, along the lines of Rivest's, that could have limited JSON's popularity, or resulted in people just ignoring parts of it and focusing on the parts they found useful.
The original JSON RFC-4627 was about 1/3rd the size of the original Rivest draft (a body of 260 lines vs. 750); it defines a single representation instead of four; and e.g. the section on "Encoding" is just 3 sentences. Here it is, for reference: https://www.ietf.org/rfc/rfc4627.txt
I’m sure you’re right. If even this simple spec exceeded what people would actually use as a real standard, surely anything beyond that would also be left by the wayside.
Great, this looks like JSON. Is it JSON5? Does it expect bigint support? Can I use escape chars?
To your specific points:
1. JSON5 didn't exist when JSON adoption occurred, and in any case they're pretty easy to tell apart, because JSON requires keys to be quoted. This is a non-problem. Why do you think it might matter? Not to mention that the existence of some other format that resembles JSON is hardly a reflection on JSON itself, except perhaps as a compliment to its perceived usefulness.
2. Bigint support is not a requirement that most people have. It makes no difference to adoption.
3. Escape character handling is pretty well defined in ECMA 404. Your point is so obscure I don't even know specifically what you might be referring to.
Readability is a function of the reader, not the medium.
oh boi, that's some Lisp-like vs C-like level of holywar you just uncovered there
and wooow my opinion is opposite of yours
You are going way out of your way to try to come up with ways to rationalize why JSON was a success. The ugly truth is far simpler than what you're trying to sell: it was valid JavaScript. JavaScript WebApps could parse JSON with a call to eval(). No deserialization madness like XML, no need to import a parser. Just fetch a file, pass it to eval(), and you're done.
Not really. JSON's mass adoption is tied to JavaScript's mass adoption, where sheer convenience and practicality dictates it's whole history most of the current state. Sending JavaScript fragments from the backend is a technique that didn't really stopped being used just because someone rolled out a JSON parser.
I think some people feel compelled to retroactively make this whole thing more refined and elegant because for some the ugly truth is hard to swallow.
Perhaps, but it's not a major concern when you control both the JavaScript frontend and whatever backend it consumes. In fact, arguably this technique is still pretty much in use today with the way WebApps get a hold of CSRF tokens. In this scenario security is a lesser concern than, say, input validation.
You don't do that, any more than you read or write machine code in binary. You read and write regular S-expressions (or assembly code) and you translate that into and out of canonical S expressions (or machine code) with a tool (an assembler/disassembler).
Reading happens even more times. I am constantly printing out API responses when I am coding, verifying what I am seeing matches what I am expecting, or trying to get an idea of the structure of something. Sure, you can tell me I shouldn’t do this and I should just read a spec, but in my experience it is often much faster just to read the JSON directly. Sometimes the spec is outdated, just plain wrong, or doesn’t exist. Being able to read the JSON is a regular part of my day.
https://en.wikipedia.org/wiki/S-expression
A canonical S-expression is a binary format, intended to be both generated and parsed by machines, not humans:
In practice, "alg:none" is a headache and everyone involved should be ashamed.
JSON is better than XML, but it really isn’t great.
JSON doesn't even support comments, c'mon. I mean, it's handy for some things, but I don't know if I'd say "JSON is better than XML" in any universal sense. I still go by the old saw "use the right tool for the job at hand". In some cases maybe it's JSON. In others XML. In others S-Exprs encoded in EBCDIC or something. Whatever works...The problem with your solution is that it’s also not portable for the same reason (it’s not part of the standard), and the reason that it wasn’t done that way in the first place is because it wouldn’t map to those initial js types!
FYI, you can easily work around this by using replacer and revivers that are part of the standards for stringify and parse and treat numbers differently. But again, the json isn’t portable to places without those replacer/revivers.
I.e, the real problem is treating something that looks like json as json by using standards compliant json parsers - not the apparent structure of the format itself. You could fix this problem in an instant by calling it something other than JSON, but people will see it and still use a JSON parser because it looks like JSON, not because it is JSON.
Of course, if you're doing this in JS and have reasons to think the resulting number may be larger than the precision of a double, you have a huge problem either way. Just as you would if you were writing this in C and thought the number may be larger than what can fit in a long long. But that's true regardless of how you represent it in JSON.
https://www.php.net/manual/en/language.types.numeric-strings...
The DER encoding used in the TLS certificates uses the big endian binary format. OpenSSL API wants the big endian binary too.
The format used by this protocol is a simple one.
It's almost exactly the format that is needed to use these numbers, except JSON can't store binary data directly. Converting binary to base 64 is a simple operation (just bit twiddling, no division), and it's easier than converting arbitrarily large numbers between base 2 and base 10. The 17-bit value happens to be an easy one, but other values may need thousands of bits.
It would be silly for the sender and recipient to need to use a BigNum library when the sender has the bytes and the recipient wants the bytes, and neither has use for a decimal number.
Kinda blows my mind that the accepted behavior is to just overflow and not raise an exception.
I try to stick to strings for anything that's not a 32 bit int now.
hard disagree on that one.
Ok, thanks for your insight.
Probably there are types not every parser/language can accept, but at least it could throw a meaningful error instead of guessing or even truncating the value.
Haskell’s Aeson library is one of the few exceptions I’ve seen, since it only parses numbers to ‘Scientific’s (essentially a kind of bigint for rationals.) This makes the API very safe, but also incredibly annoying to use if you want to just munge some integers, since you’re forced to handle the error case of the unbounded values not fitting in your fixed-size integer values.
Most programmers likely simply either don’t consider that case, or don’t want to have to deal with it, so bad JSON libraries are the default.
There's an element of "worse is better" here [1]. JSON overtook XML exactly because it's so simple and solves for the social element of communication between disparate projects with wildly different philosophies, like UNIX byte-oriented I/O streams, or like the C calling conventions.
---
[0] https://ecma-international.org/publications-and-standards/st...
JSON could easily be extended to support them - but there’s no standards body with the authority to make a change like that. So we’re probably stuck with json as-is forever. I really hope something better comes along that we can all agree on before I die of old age.
While we’re at it, I’d also love a way to embed binary data in json. And a canonical way to represent dates. And comments. And I’d like a sane, consistent way to express sum types. And sets and maps (with non string keys) - which JavaScript also natively supports. Sigh.
JSON is a victim of its success: it has become too big to fail, and too big to improve.
The fact remains that json doesn’t have native support for any of this stuff. I want something json-like which supports all this stuff natively. I don’t want to have to figure out if some binary data is base64 encoded or hex encoded or whatever, and hack around jackson or serde or javascript to encode and decode my objects properly. Features like this should be built in.
In code you control you can choose to use JSON5: https://json5.org/
JSON is still hack garbage compared to XML from the turn of the millennia. Like most dominant tech standards, JSON took hold purely because many developers are intellectually lazy and it was easier to slam some sloppy JSON together than to understand XML.
XML with XSD, XPath and XQuery is simply a glorious combination.
Oh JSON.
For those unfamiliar with the reason here, it’s that JSON parsers cannot be relied upon to treat numbers properly. Is 4723476276172647362476274672164762476438 a valid JSON number? Yes, of course it is. What will a JSON parser due with it? Silently truncate it to a 64-bit or 63-bit integer, or a float, probably or if you’re very lucky emit an error (a good JSON decoder written in a sane language like Common Lisp would of course just return the number, but few of us are so lucky).
So the only way to reliably get large integers into and out of JSON is to encode them as something else. Base64-encoded big-endian bytes is not a terrible choice. Silently doing the wrong thing is the root of many security errors, so it not wrong to treat every number in the protocol this way. Of course, then one loses the readability of JSON.
JSON is better than XML, but it really isn’t great. Canonical S-expressions would have been far preferable, but for whatever reason the world didn’t go that way.