News for hackers. Stuff that matters.
Re:Slimmer and faster JavaScript strings in Firefox by iopq on Monday July 21, 2014 @09:29PM (#8067034)
That's a bad interface that allows you to split strings at useless codepoints and get illegal UTF-16 strings as the result.
Re:Slimmer and faster JavaScript strings in Firefox by dbaupp on Tuesday July 22, 2014 @01:25AM (#8067502)
It's the historical interface which websites now rely on, changing it would be like writing a libc with strcmp operating on Pascal strings.

In any case, a Javascript String is not actually designed to be UTF-16, it is essentially just an `uint16_t[]`. Even textual strings just store UTF-16 code units, not full UTF-16 data. Relevant snippets from the standard:

The String type is the set of all finite ordered sequences of zero or more 16-bit unsigned integer values ("elements").

When a String contains actual textual data, each element is considered to be a single UTF-16 code unit. [...] All operations on Strings (except as otherwise stated) treat them as sequences of undifferentiated 16-bit unsigned integers; they do not ensure the resulting String is in normalised form, nor do they ensure language-sensitive results.

See also:

- Section 8.4 http://www.ecma-international.org/publications/files/ECMA-ST...

- http://mathiasbynens.be/notes/javascript-encoding

Re:Slimmer and faster JavaScript strings in Firefox by gsnedders on Tuesday July 22, 2014 @01:55AM (#8067551)
> Although the standard does state that Strings with textual data are supposed to be UTF-16.

No, it doesn't. It states that they're UTF-16 code units, a term defined in Unicode (see D77; essentially an unsigned 16-bit integer), which is not the same as UTF-16. A sequence of 16-bit code units can therefore include lone surrogates, which something encoded in UTF-16 could not.

Re:Slimmer and faster JavaScript strings in Firefox by dbaupp on Tuesday July 22, 2014 @01:57AM (#8067554)
Oh, yes; I just skimmed 'code unit' bit without actually reading. (I've now removed the misinformation from my previous comment.)
Re:Slimmer and faster JavaScript strings in Firefox by TazeTSchnitzel on Monday July 21, 2014 @09:46PM (#8067071)
I think JS may be from the time when UCS-2 was all there was and there were only 65535 Unicode characters.
Re:Slimmer and faster JavaScript strings in Firefox by pcwalton on Monday July 21, 2014 @09:46PM (#8067069)
It's needed for compatibility with the Web, unfortunately.
Re:Slimmer and faster JavaScript strings in Firefox by loganfsmyth on Tuesday July 22, 2014 @12:22PM (#8069766)
Definitely. Thankfully, ES6 will introduce

    "🍣".codePointAt(0)
and iterators will iterate code points, not code units.

    for (var c of "🍣"){}