Re:How To Use UTF-8 Throughout Your Web Stack
by zokier on Saturday August 20, 2011 @12:05AM
(#2905690)
The error does not come from “Iñtërnâtiônàlizætiøn”, but from “Iñtërnâtiônà lizætiønâ€, which contains U+009D character (correctly encoded and delivered as UTF-8). Apparently that character is not allowed in HTML documents. So & is not the only character you need to escape, and seeing that even you didn't know this detail, I don't think it's a bad habit to play on the safe side and just escape all non-ascii (printable) characters.
edit: I read the HTML5 spec, it says: "Text must not contain control characters other than space characters". So a reasonable solution would be to pass all printable characters as UTF8 and encode control characters. But as I said, I'd prefer to err on the side of caution, in this case encode more than necessary if I'm not sure exactly which characters need encoding and which do not.
Re:How To Use UTF-8 Throughout Your Web Stack
by pavpanchekha on Saturday August 20, 2011 @01:03PM
(#2907062)
No, your problem is that the UTF-8 encoding of U+009D isn't 9d, it's c2 9d. So if you're encoding it as 9d, you're not writing out UTF-8, you're writing out latin-1, which of course leads to displaying random characters. Serve your page as utf-8 and encode it properly.
If your page is really being delivered as UTF-8 then it will pass all validation, just using the real characters. (You still have to escape & as & of course.)
Here, I put together a little example for you: http://50pop.com/i18n.html
View source to verify. Click the validate link.
Hope that helps.