"It was definitely a mistake on our part. The problem was not the encoding on our data feed, but the HTML document was sent with ISO-8859-1. The second we inserted the twitter text into the DOM, the browsers interpreted the UTF-8 string as ISO-8859-1. Our visualizations are hosted on other platforms, and in this case the server was not configured to send UTF-8 with text/html even though the HTML file was encoded as such. It was the only issue (albeit a pretty obvious one) during an otherwise flawless event. I apologize to President Obama, Speaker Boehner, and Jack Dorsey for the mistake. If the readers of the blog think it was stupid, imagine how we felt. dev environment != production environment. If we would have just included a <meta charset="utf-8"> in the HTML head, then this would not have occurred.
The big take away is don’t make assumptions about other platforms (especially when it comes to encoding), and always include charset meta tag." [emphasis mine]
Something like this could be done as nginx/apache module: which detects encodings of the data and transcodes the HTML output into utf8 - could be useful for some cases.
Every time you don't validate...God kills a kitten. Please, think of the kittens.
This has been a Public Service Announcement. Please code responsibly: http://validator.w3.org/
<meta charset="utf-8">Edit: yes, there were definitely people curating the questions, so it could not have been as simple as pulling one JSON feed. http://www.theatlantic.com/politics/archive/2011/07/how-obam... https://twitter.com/#!/townhall/july-6-curators
...That's kind of the point. There was someone being sloppy there.
That's a sloppy mistake and should without doubt have been caught in testing.