Re:Unicode is, in 2012, still “cutting edge.”
by ori_b on Tuesday July 10, 2012 @04:37PM
(#4226027)
There's no good reason for one to be faster than the other, though! They're both utf8 encoded sequences of bytes, and there's no good reason to not stream through them as utf8.
[ Parent
| on HN ]
> There's no good reason for one to be faster than the other, though!
Not exactly. A varchar will store it as is, but a char column will allocate a fixed 3 (or 4) bytes for each character.
All data stored in memory (for sorting and such) is always as char, even if it started as varchar.
So by allowing 4 bytes per character they use more memory.
Re:Unicode is, in 2012, still “cutting edge.”
by excuse-me on Tuesday July 10, 2012 @04:58PM
(#4226160)
Internally it can store them as 16bit so strings have a fixed length
Re:Unicode is, in 2012, still “cutting edge.”
by pjscott on Tuesday July 10, 2012 @06:07PM
(#4226491)
Does anybody really think that UCS-2 is a good idea anymore? Or that random indexability by code point is all that valuable, in a world with combining glyphs and bidirectional characters and whatever other crazy stuff Unicode has? If you just want an upper bound on the number of bytes needed to store n code points, then (a) that's probably not a particularly useful question to ask, and (b) if you assume that 32 bits is enough for any code point, then the space taken by properly-formed UTF-8 is bounded.
So, why would they want to store things internally as UCS-2? Or rather, why should they?
Which is in many cases not faster than UTF-8 since UTF-16 is often more bytes than UTF-8. This matters especially since we are talking about a database which means IO and RAM usage probably are more important than the CPU.
Re:Unicode is, in 2012, still “cutting edge.”
by Dylan16807 on Tuesday July 10, 2012 @08:17PM
(#4227023)
Except it didn't. It used up to three bytes per character. It had the drawbacks of variable width but not the easy-to-add benefits.
Or do you mean in-memory being different from the file format AND different from the I/O format? That doesn't sound terribly efficient.