News for hackers. Stuff that matters.
Re:PHP Doesn't Suck Anymore by gbuk2013 on Saturday May 04, 2024 @06:34AM (#40256507)
No, JS strings are UTF-8:

    > '蛋糕'.substr(0,1)
    '蛋'
    > '蛋糕'.length
    2
    > Buffer.byteLength('蛋糕')
    6
You do have to be careful when working with binary data (e.g. streams) but this is expected.
Re:PHP Doesn't Suck Anymore by njuw on Saturday May 04, 2024 @07:40AM (#40256891)
They're UTF-16, and substr(), length, etc, work at the code unit level. Hence, the above isn't actually valid for all strings - any characters that are represented by codepoints between U+10000 and U+10FFFF require 2 code units [1]. For example U+10429 Deseret Small Letter Long E [2]

  > '𐐩'.substr(0, 1)
  '\ud801'
  > '𐐩'.length
  2
[1] https://en.wikipedia.org/wiki/UTF-16#Description

[2] https://codepoints.net/U+10429

Re:PHP Doesn't Suck Anymore by gbuk2013 on Saturday May 04, 2024 @08:08AM (#40257045)
TIL but it just proves my point all the more that it's the wrong level of abstraction for PHP considering what it is used for and who the target users are. :) I shouldn’t need to care about this in 2024 when building web pages.
Re:PHP Doesn't Suck Anymore by gbuk2013 on Saturday May 04, 2024 @01:50PM (#40259183)
TIL thanks :) Interestingly, "for of" iteration works on the whole character, so must be some magic going on under the hood.
Re:PHP Doesn't Suck Anymore by trurl42 on Saturday May 04, 2024 @07:04AM (#40256657)
And with that you're completely wrong, since strings in JavaScript are UTF-16.

It just so happens that your example consists of two UTF-16 codepoints.

(Node.js' Buffer uses UTF-8 by default).

Re:PHP Doesn't Suck Anymore by kiitos on Sunday May 05, 2024 @08:53PM (#40270088)
One ambiguity here might be that Javascript defines strings as UTF-16, but JSON defines strings as UTF-8.
Re:PHP Doesn't Suck Anymore by DonHopkins on Saturday May 04, 2024 @08:08AM (#40257040)
The 蛋糕 is a lie!