News for hackers. Stuff that matters.
Re:How To Use UTF-8 Throughout Your Web Stack by zokier on Saturday August 20, 2011 @12:31AM (#2905736)
I agree on most of your points, but disagree on that guessing encoding should not be done. I think that it conflicts with basic robustness principle "be conservative in what you do, be liberal in what you accept from others".
Re:How To Use UTF-8 Throughout Your Web Stack by sid0 on Saturday August 20, 2011 @05:01AM (#2906223)
I personally think being liberal in what you accept from others is the second worst evil in computer science. The worst being null, of course.
Re:How To Use UTF-8 Throughout Your Web Stack by tlrobinson on Saturday August 20, 2011 @06:23AM (#2906316)
I agree. It allows sloppy developers to be liberal in what they do, and leads to increasingly complex (and incompatible) implementations necessary to be compatible with all the edge cases.

HTML is a good example. Browsers are very tolerate of malformed HTML, which is nice for beginners who don't want to worry too much about perfect syntax.

The problem is each browser handles the unspecified cases differently, which leads to differences in the way pages are rendered, security issues like XSS, etc.

Robustness should just be built into the protocol/format/spec, if necessary. HTML5 gets this right by specifying an algorithm that all parsers should use to get consistent behavior, while still being tolerant of imperfect syntax: http://en.wikipedia.org/wiki/Tag_soup#HTML5

Re:How To Use UTF-8 Throughout Your Web Stack by jrockway on Saturday August 20, 2011 @06:14AM (#2906305)
Hey now. If software started validating its input, what would virus writers do for a living?
Re:How To Use UTF-8 Throughout Your Web Stack by guard-of-terra on Saturday August 20, 2011 @05:36AM (#2906263)
Then you will also personally produce programs which would be broken for 5/6ths of the world population who happen to use letters outside latin1.

There's no way to avoid it unless you wrap it up and add some explicit checks and guesses.

Re:How To Use UTF-8 Throughout Your Web Stack by tlrobinson on Saturday August 20, 2011 @06:08AM (#2906298)
Won't all modern browsers include the encoding in the Content-Type header?

They should. If so there's no need to guess.

Re:How To Use UTF-8 Throughout Your Web Stack by guard-of-terra on Saturday August 20, 2011 @09:25AM (#2906581)
It's not just browsers. Browsers are pretty sane when it comes to charsets, because they had the time to make it right and the pressure to do so. (It wasn't like that in times of NN4/IE4, which would interpret your text as whatever they want and won't even let you override)

Facing to something less agamant (like dreaded id3 tags), no such luck.