News for hackers. Stuff that matters.
Re:Hacker News RSS Feed + Readability by SomeIdiot on Wednesday April 01, 2009 @06:50PM (#542592)
Yeah, apostrophes seem to turn into some wierd jumble (e.g. ’) and so do other charaters. Aside from that, very nice work. Added to my reader.
Re:Hacker News RSS Feed + Readability by nirmal on Wednesday April 01, 2009 @07:13PM (#542633)
Any clues as to why that happens? The code simply inserts the HTML of the content sections of the linked sites into the RSS feed. It doesn't seem to happen with all sites.
Re:Hacker News RSS Feed + Readability by bd on Wednesday April 01, 2009 @07:40PM (#542662)
Probably it happens when the real text encoding is different from the declared one (at least that's the problem I encountered when aggregating content from unfiltered wild web).

If it really bothers, you can use chardet [1] to try to detect the real encoding (BeautifulSoup should use it if it's installed). But even this is not 100% foolproof.

[1] http://chardet.feedparser.org/

Re:Hacker News RSS Feed + Readability by jauco on Thursday April 02, 2009 @02:24AM (#542982)
Yep, ’ means that the pages contains the UTF-8 quote character, but the browser renders the bytestream as if it's a single byte character stream.

The crux is that the basic alphabet is encoded the same. So you only notice it with special characters such as the curly quote and the em-dash.