UTF-8 Everywhere (2012)

Re:UTF-8 Everywhere (2012) by paulddraper on Monday June 20, 2016 @09:19PM (#11942869)

His point was that the transfer format should be conceptually independent of the processing. Obviously it has to be encoded in RAM in some way, but the programmer doesn't need to worry about the memory layout.

[ Parent | on HN ]

Re:UTF-8 Everywhere (2012) by Avernar on Monday June 20, 2016 @11:36PM (#11943311)

Why? Why should it be conceptually different if it's easy to work with the encoded form?

Many unicode languages work with UTF-8 or UTF-16 internally. So working with the "transfer format" is common practice.

While it may not be necessary to know who the languages your program in work under the hood, expert programmers do want/need to know. That way they can write better code, or switch to another language or get the language devs to improve their internal handling.

[ Parent | on HN ]

Re:UTF-8 Everywhere (2012) by paulddraper on Tuesday June 21, 2016 @12:04PM (#11946719)

Think of JSON.

The programmer shouldn't have to know that a newline character is written as \n in a JSON string.

The JSON string "a\nb" take 6 characters to write, but it's length should be given as 3.

99% people want to manipulate a JSON model, not the JSON (or BSON) serialization itself. The 1% can still use a byte array and do whatever hacks they like.

[ Parent | on HN ]

Re:UTF-8 Everywhere (2012) by Avernar on Tuesday June 21, 2016 @01:41PM (#11947662)

Bad example. If you want to embed that string in your code you have to type those 6 characters anyways.

A better example is if you want to find a newline in a string. If you do a find it in a UTF-16 string it may be position 8 and a find in UTF-8 may be position 12. Does it matter what the actual number is? NO. You just pass it to the next function or whatever.

[ Parent | on HN ]