Re:Lessons of JSON
by pixelcort on Monday April 30, 2012 @01:22PM
(#3910241)
The other limitation of JSON is there doesn't appear to be a standard way of representing Unicode code points above 16 bits (such as some Emoji) in strings. One way I've seen it done is to put two escaped UTF-16 surrogate pairs together; another way I've seen it done is having the UTF-8 literal inside the string.
[ Parent
| on HN ]
Please forgive my ignorance, but why not? Couldn't the entire JSON response be, for instance, UTF-8 encoded, thus allowing the representation of all possible Unicode code points?
Yes. In fact, JSON documents must be Unicode text, and the default encoding is UTF-8.
Re:Lessons of JSON
by function_seven on Monday April 30, 2012 @01:35PM
(#3910301)
I'm still wet behind the ears when it comes to Unicode, but why can't the character just be transmitted "natively"? (Like you said, UTF-8 inside the string assuming the encoding is set to utf8) {"I'm unicode" : "←"}
In case anybody is nervous about this, the JSON RFC declares UTF-8 to be the default encoding, and points out a simple rule for reliably detecting the Unicode encoding (and endianness) of any JSON text encoded with UTF-8, UTF-16, or UTF-32. You can stick any Unicode characters in a JSON document, unescaped, with confidence that any halfway decent parser will decode it properly.
Apparently the first solution is the correct according to the RFC: http://www.ietf.org/rfc/rfc4627.txt
To escape an extended character that is not in the Basic Multilingual Plane, the character is represented as a twelve-character sequence, encoding the UTF-16 surrogate pair.
This is regardless of whether the json itself is UTF-8 or UTF-16 encoded.