News for hackers. Stuff that matters.
Re:The UTF-8-Everywhere Manifesto by fauigerzigerk on Sunday April 29, 2012 @01:30PM (#3906445)
still, text for today's computer capabilities is a non issue even if using UTF-32

That obviously depends entirely on what kind of application we're talking about. Keeping large amounts of text data in memory as efficiently as possible is one of my greatest concerns. Many people are processing lots of text nowadays, more than ever before.

"It's not fixed width" But for most text, it is.

True, so ignoring it means that your code will be correct ... most of the time.

Re:The UTF-8-Everywhere Manifesto by raverbashing on Sunday April 29, 2012 @02:16PM (#3906571)
"Keeping large amounts of text data in memory as efficiently as possible is one of my greatest concerns"

Depends on what you consider efficiency. If it's size, sure, store it using UTF-8. But if you're worried about speed, then UTF-16 or 32 may be the way to go, since you're dealing with data that fits a CPU register. For example, on ARM comparing one byte is much more work than comparing one 32-bit value.

"True, so ignoring it means that your code will be correct ... most of the time."

No, not going to ignore it! But on UTF-16 more code points match the UTF-16 encoding of it (easier for debugging)