News for hackers. Stuff that matters.
Re:The UTF-8-Everywhere Manifesto by natch on Sunday April 29, 2012 @03:02PM (#3906686)
Strings (NSString) on Apple platforms are UTF-16. The Apple platforms are not exactly lagging behind in either multilingual, or text processing. I wonder what this team of three people knows that Apple doesn't? Or is it the other way around, that Apple knows something they don't, and when it comes to shipping products that work in the real world, Apple has figured out how to do it?
Re:The UTF-8-Everywhere Manifesto by __david__ on Sunday April 29, 2012 @03:12PM (#3906732)
NSStrings are opaque--you always call accessor functions and never have access to the low level backing store. The reason they are good is that you can't get data into or out of them without specifying an encoding, which leaves the actual encoding of the backing store as an implementation detail.

The fact is, I don't even know (or see documented) that the backing is UTF-16--Apple is free to change that at their whim and no user programs would break.

Re:The UTF-8-Everywhere Manifesto by brigade on Sunday April 29, 2012 @05:51PM (#3907250)
It's not documented (presumably) for that very reason.

In fact, the opposite is implied by initWithBytesNoCopy:length:encoding:freeWhenDone: - it should be possible right now to have NSStrings with arbitrary internal representations, even if most other creation methods currently convert to UTF16.

Re:The UTF-8-Everywhere Manifesto by cobbal on Sunday April 29, 2012 @05:59PM (#3907266)
Unfortunately, this isn't entirely true.

  - (unichar)characterAtIndex:(NSUInteger)index
will return the UTF-16 "characters" (codepoints or half-codepoints)
Re:The UTF-8-Everywhere Manifesto by Xuzz on Sunday April 29, 2012 @03:06PM (#3906705)
NSString is decades old, from NeXTSTEP (you can see that in the name: "NS"). While its possible they could change it, the in-memory representation doesn't matter much in this case. When you transfer data out, such as with writeToFile:encoding: or convert it into a char * (often with UTF8String, or one of the C string methods), you are almost always specifying an encoding anyway. And, for most Cocoa apps, that encoding is UTF8.
Re:The UTF-8-Everywhere Manifesto by gregschlom on Sunday April 29, 2012 @03:09PM (#3906717)
As the authors explain in the post, many things use UTF-16 internally (Python, Java, C#, etc...). It does not mean that it's the best solution.

Have you read the article? They make a good explanation at why UTF-16 is "the worst of both worlds" (wide characters AND variable lenght).