News for hackers. Stuff that matters.
Re:RSS mistakes: let's not make them again by lwf on Wednesday July 17, 2013 @02:40PM (#6059389)
> PubSubHubbub evolved and is now able to work with any kind of data (not just RSS or Atom), opening the door to a JSON based syndication format.

It is not made clear why this is desirable. The author makes no case for why a JSON-based system would be any better than the current XML system. Perhaps the implied "eww XML", but XML is the standard.

Re:RSS mistakes: let's not make them again by bct on Wednesday July 17, 2013 @03:22PM (#6059654)
People are (slightly?) less likely to do the stupid "let's generate this structure by gluing strings together" thing that produced the need for liberal Atom and RSS parsers.

(But yes, I agree that "turn this already-existing thing into JSON just because" is a stupid trend.)

Re:RSS mistakes: let's not make them again by saurik on Wednesday July 17, 2013 @05:25PM (#6060285)
The thing that screwed RSS was not it being XML: I've seen the exact same issues happen any time someone has a text field in any file format. (Hell, I'm guilty of this.) They are more related to poor stewardship of a shared protocol than due to the usage of any particular transport encoding.

The first issue is that you have a field, and that field is rendered in a text box, and is defined to be text; at some point you go "man, I wish I could add a hyperlink to my text", and so you now want to put some HTML in there. However, the field is just text, so what do you do?

Let's say this was JSON, and this text is a string; do you put HTML in the string? This is somewhat equivalent to putting escaped HTML into the text node of the XML document. Alternatively, you could replace the string with an object (vaguely equivalent to putting HTML elements in the RSS).

The result of some people choosing the first option (which seems more reasonable at first glance than the second option, as it provides a better experience for existing readers and better fits the existing protocol) is having to look at a string and guess whether it should be parsed as HTML or not.

The second problem is that as people make changes in the second direction, if they are not carefully organized and centralized in a specification, you end up with haphazard and incompatible changes: someone decides to add a "type" field with a mime type, someone else adds a different element.

Having a ton of people generate the format, and having the thing parsed by some liberal canonical language, leads to too much flexibility in fields like dates: someone puts a weird date format into their date field, and it is parsed correctly, and then tons of other people do it, and you're screwed.

This also isn't helped by going with JSON: the field is just more likely to end up being "anything that a JavaScript Date object is willing to convert from a string to a date", which assuredly supports irritating corner cases that are not supported by the Date parsers from other random languages.

With RSS, it was really "the peoples' protocol", with the specifications only encoding random changes that had become popular over time. What we were left with was a total mess: I can't find it now, but someone once wrote a proof that you couldn't actually parse RSS due to conflicting standards.

Re:RSS mistakes: let's not make them again by julien on Wednesday July 17, 2013 @04:12PM (#6059970)
Many people have argued that RSS is too complex compared to JSON... and to be fair it's clear now that every new API from every service is JSON based. At this point, XML is clearly on the way out for web people!
Re:RSS mistakes: let's not make them again by bct on Wednesday July 17, 2013 @04:13PM (#6059977)
That's the same kind of attitude that caused the decade of "XML for everything, and damn the technical requirements".
Re:RSS mistakes: let's not make them again by julien on Wednesday July 17, 2013 @04:51PM (#6060142)
Agreed, and we (superfeedr) have been strong advocates of XML for some problems. We are also tired of hearing: give us JSON or we won't care about RSS.
Re:RSS mistakes: let's not make them again by ancarda on Wednesday July 17, 2013 @02:49PM (#6059448)
JSON is smaller and easier to parse in JavaScript. Aside from that, there are no real advantages IMO.
Re:RSS mistakes: let's not make them again by MatthewPhillips on Wednesday July 17, 2013 @03:03PM (#6059541)
Parsing RSS/Atom is a solved problem, don't do it yourself, use libraries.
Re:RSS mistakes: let's not make them again by finnw on Wednesday July 17, 2013 @04:13PM (#6059976)
Parsing JSON is also a solved problem. And JSON is also easier for humans to read than XML.
Re:RSS mistakes: let's not make them again by bct on Wednesday July 17, 2013 @04:22PM (#6060017)
> JSON is also easier for humans to read than XML.

That's extremely debatable.

Re:RSS mistakes: let's not make them again by ancarda on Wednesday July 17, 2013 @04:22PM (#6060013)
Writing JSON is also easier by hand than XML, or at-least it is for me. Any time I'm planning a data structure I always use JSON since it's so well structured.

XML is quite loose, unless I use plist theres no data types which is both an advantage and a disadvantage.

Re:RSS mistakes: let's not make them again by jarek on Wednesday July 17, 2013 @04:16PM (#6059992)
But is parsing an RSS-like format that uses JSON a solved problem?
Re:RSS mistakes: let's not make them again by dragonwriter on Wednesday July 17, 2013 @04:36PM (#6060079)
That's just parsing JSON and interpreting the data. Since parsing JSON and producing arbitrary object models is a solved problem, parsing an x-like format that uses JSON is also a solved problem for all x (developing the object model to use for it is an issue, perhaps, but you have to have that before you can have any serialization format, so that's not a distinguishing problem between different approaches.)

Note that I am not, here, endorsing JSON over other alternatives; the same is obviously true of XML, HTML, YAML, or lots of other potential formats.

Re:RSS mistakes: let's not make them again by jarek on Wednesday July 17, 2013 @05:20PM (#6060259)
Right, the only point I'm making is that ease of parsing JSON shouldn't count as a factor against RSS since RSS parsing is already more done than parsing any JSON-powered RSS alternative as the object model is already defined and implemented.
Re:RSS mistakes: let's not make them again by zanny on Wednesday July 17, 2013 @01:10PM (#6058821)
> For RSS, I have first to “guess” (or rather hope) that there is a feed, since browsers now hide the infamous orange icon. I can then select the url of that tab and copy it. Then, open a new tab, go to the reader I chose to use and paste the url. Hopefully, that reader is smart enough to actually find the feed url from this page’s url. If not, I’m screwed anyway and I’ll have to look into the HTML code of the page!

On sites that provide a feed link, I just click that link, which opens that feed in feedly, and I hit add to my feedly. Process is as simple as twitter. The choice of some sites to hide their feeds and provide no links is the burden of site designers.

Re:RSS mistakes: let's not make them again by jessaustin on Wednesday July 17, 2013 @01:16PM (#6058866)
If one doesn't like how her browser handles certain links, it's easy enough to get an extension or bookmark for that browser that does exactly what she wants. This is especially true for feeds.
Re:RSS mistakes: let's not make them again by julien on Wednesday July 17, 2013 @04:14PM (#6059978)
Do you really think people who don't know what their browser is will install an extension or a bookmarklet?... Again, compare that experience to a click on "follow me on twitter" button.
Re:RSS mistakes: let's not make them again by mpweiher on Thursday July 18, 2013 @01:37AM (#6061952)
I don't have to install any extensions in my browser. It Just Works™.
Re:RSS mistakes: let's not make them again by th0ma5 on Wednesday July 17, 2013 @02:15PM (#6059224)
Re:RSS mistakes: let's not make them again by sentenza on Wednesday July 17, 2013 @01:29PM (#6058942)
What browsers actually hide the rss icon?

I too sometimes have to resort to the tactics that the article describes to get a feed, but it is usually just wordpress sites where the author is unaware that rss even exists or that rss is provided by wordpress.

Re:RSS mistakes: let's not make them again by eli on Wednesday July 17, 2013 @01:32PM (#6058955)
I believe both Chrome and Firefox used to show an RSS icon next to the URL but both killed the feature.

Here's a Chrome plugin to add it back: https://chrome.google.com/webstore/detail/rss-subscription-e...

Re:RSS mistakes: let's not make them again by sentenza on Wednesday July 17, 2013 @01:42PM (#6059021)
Ah, thank you. I am using Firefox for so long now that I can't remember when I started and somehow never noticed that the icon was there at some point.
Re:RSS mistakes: let's not make them again by eli on Wednesday July 17, 2013 @01:47PM (#6059054)
Firefox's was actually pretty good in that you could click it and it made an attempt to format the feed for display and offered you links to subscribe to it using popular Feed Readers.

Chrome, even when it had the icon, by default linked it to the unformatted XML page. I get the sense that Google's heart was never really in RSS.

Re:RSS mistakes: let's not make them again by Spittie on Wednesday July 17, 2013 @02:37PM (#6059374)
Firefox still show that page, you just have to open the feed directly in the browser (instead of pressing that orange icon).

You can put the icon back in the toolbar, or you can press on the globe/keylock near the url, press on more informations, and then Firefox will show all the available feeds for the page under the "feeds" tab.

Re:RSS mistakes: let's not make them again by nieve on Wednesday July 17, 2013 @01:58PM (#6059124)
You can put the RSS icon back in the toolbar (but not the url bar) by going into View->Toolbars->Customize and dragging it wherever you want it. It's a bit less obvious than the original, but it's still useful.
Re:RSS mistakes: let's not make them again by julien on Wednesday July 17, 2013 @07:12PM (#6060648)
Good luck with getting anyone who isn't a nerd to do it!
Re:RSS mistakes: let's not make them again by nieve on Thursday July 18, 2013 @12:10AM (#6061733)
Yep. I offered the info in case it was useful, but I don't think it lets Mozilla off the hook at all. It's slightly better than completely getting rid of the feed detection code, I guess. That's probably the next step...
Re:RSS mistakes: let's not make them again by julien on Wednesday July 17, 2013 @04:13PM (#6059973)
Yes, feedly is a browser extension, that you knew how to install Most people don't and most readers are not browser extensions...
Re:RSS mistakes: let's not make them again by zanny on Wednesday July 17, 2013 @06:23PM (#6060460)
I don't actually use the feedly browser extension, it mucks my pages up. I use normal web page feed links, and Firefox (the first time) will ask where you want to direct that rss feed, and if you have feedly open in a tab it proposes feedly as an option, and you can set it as the default so links always go to feedly. No extension required.
Re:RSS mistakes: let's not make them again by th0ma5 on Wednesday July 17, 2013 @12:53PM (#6058714)
Oblig XKCD: http://xkcd.com/927/

I think also however that this post brings up a lot of good points. Specifically I feel it touches on a very important one subtly... we do have now a "web of data" even if it isn't exactly the W3C spec of such a thing, we still have one. Now what?

Re:RSS mistakes: let's not make them again by haakon on Wednesday July 17, 2013 @01:12PM (#6058831)
It's too bad FeedTree (http://www.feedtree.net/) never caught on. Decentralised, peer-to-peer push distribution of RSS-style updates. Instead we ended up either hammering servers checking for updates, or using centralized hubs like FeedBurner, and eventually moving to completely proprietary services like Twitter and Google+.
Re:RSS mistakes: let's not make them again by wmf on Wednesday July 17, 2013 @03:39PM (#6059766)
A feed is fundamentally centralized anyway, so I don't see the problem with relying on the feed to push updates to you. Unfortunately because Google build a super-centralized crutch into PuSH everybody thought mooching off Google was the point of PuSH so sites never installed their own hubs.
Re:RSS mistakes: let's not make them again by mpweiher on Wednesday July 17, 2013 @03:51PM (#6059846)
Hmm...when I click on an RSS feed link in Safari, my RSS reader (NetNewsWire) opens and asks me if I want to add this feed. (Worked for the comments feed, didn't see a link for the blog, and the "subscribe" button didn't work).

Yes, polling has issues, but the author doesn't explain how changing the file format from RSS to some JSON-based format makes those problems go away.

Maybe I am just dense, but neither does the author show real problems, nor does he offer solutions. Well, the latter being somewhat unsurprising considering the former.

Re:RSS mistakes: let's not make them again by jrochkind1 on Wednesday July 17, 2013 @05:20PM (#6060261)
The “level 0” solution is to periodically fetch each feed, parse it, diff it and hopefully find something new.

Um, less solutions to this exist, but they are much simpler than 'PubSubHubbub or RSSCloud'. And don't require any changes to RSS.

You are requesting this thing over HTTP. You (and the server delivering the feed) simply need to use standard HTTP caching headers. etags, last-modified, etc.

No need to fetch the XML and parse it and diff it.

Re:RSS mistakes: let's not make them again by julien on Wednesday July 17, 2013 @07:12PM (#6060644)
I don' see how using header will remove the need to fetch the resources, parse them and diff them. Maybe you'll do that less often, but you'll still do it a lot...
Re:RSS mistakes: let's not make them again by jrochkind1 on Friday July 19, 2013 @12:15AM (#6068297)
Are you familiar with how HTTP caching headers work?

From an etag in the header, you can tell if the current remote resource is identical to the one you have locally or not, without any fetching/parsing/diffing.

Same with from a Last-Modified header, right?

In fact, you send a conditional GET rather than first retrieving only a HEAD and then deciding whether to continue with a full GET, but either way you aren't fetching/parsing/diffing to tell if there's new content.

Re:RSS mistakes: let's not make them again by stephen_mcd on Wednesday July 17, 2013 @01:51PM (#6059079)
The main criticism here doesn't really apply to any readers that apply the slightest effort in making the subscription process streamlined.

Take https://kouio.com for example (a Google Reader replacement I've built), in kouio you just enter a website's address and it'll discover the feed automagically.

Re:RSS mistakes: let's not make them again by seanzieapples on Wednesday July 17, 2013 @02:44PM (#6059425)
I like your app so far but I input https://news.ycombinator.com/ and it's not able to locate an RSS feed (Just says "Loading"). I input https://news.ycombinator.com/rss and it works.
Re:RSS mistakes: let's not make them again by stephen_mcd on Wednesday July 17, 2013 @03:44PM (#6059804)
Ha!

Well as jerf mentioned below, HN doesn't correctly provide a link reference to its RSS feed. Now that's fine in our case, as a surprisingly large number of websites fail this test, and simply have a visible link with "rss" or "atom" in either the link's text or URL.

Now we pick these up, but in this case the front page of HN has an invalid link with "rss" in the text, namely this very thread itself. So normally the HN front page URL works fine, but right now it doesn't. This gives us a great chance to refine our discovery code a bit, so thank you!

Re:RSS mistakes: let's not make them again by malkarouri on Thursday July 18, 2013 @04:11AM (#6062420)
That is one of the coolest bugs I came across lately.
Re:RSS mistakes: let's not make them again by jerf on Wednesday July 17, 2013 @03:22PM (#6059647)
HN does not include the proper metadata to indicate an RSS feed. If

    <link rel="alternate" type="application/rss+xml" 
          title="Hacker News" 
          href="http://news.ycombinator.com/rss" />
was added to the <head> of the homepage, the discovery would probably work better.
Re:RSS mistakes: let's not make them again by stephen_mcd on Wednesday July 17, 2013 @03:39PM (#6059777)
Thank you! That's definitely a bug, looking into it :-)
Re:RSS mistakes: let's not make them again by monksy on Wednesday July 17, 2013 @03:27PM (#6059682)
Google Reader did attempt to find the feed's address.
Re:RSS mistakes: let's not make them again by MatthewPhillips on Wednesday July 17, 2013 @03:01PM (#6059526)
> The “level 0” solution is to periodically fetch each feed, parse it, diff it and hopefully find something new.

Wait, do feed readers really not do a HEAD request first and check Last-Modified ?

Re:RSS mistakes: let's not make them again by bct on Wednesday July 17, 2013 @03:24PM (#6059662)
I would hope that they wouldn't do it that way; they should be sending a GET with If-Modified-Since. The same result, making half the requests.
Re:RSS mistakes: let's not make them again by gboudrias on Wednesday July 17, 2013 @03:10PM (#6059585)
I don't think those values are always reliable.
Re:RSS mistakes: let's not make them again by jrochkind1 on Friday July 19, 2013 @12:16AM (#6068302)
So which is easier, getting servers to fix them to be reliable, or getting everyone to change to a new json pubsubhub whatever thingy, that they'll probably do unrealiably anyway too?
Re:RSS mistakes: let's not make them again by julien on Wednesday July 17, 2013 @04:08PM (#6059953)
HEAD is still a waste of resources. the http body in the context of feeds is usually pretty small and does not consume a lot of resources compared to actually initiating (or even keeping) the TCP connection. HEAD also means the server as to do everything but send the data to make sure it does not need to sent it => waste.
Re:RSS mistakes: let's not make them again by icebraining on Wednesday July 17, 2013 @06:55PM (#6060566)
HEAD also means the server as to do everything but send the data to make sure it does not need to sent it

Not at all. If you have the RSS as a static file - and that's the best way for a resource that is much more often read than written - the web server can just stat() the file and get the modified timestamp instead of having to open and read it.

Re:RSS mistakes: let's not make them again by julien on Wednesday July 17, 2013 @07:13PM (#6060654)
Show me a service that has static files as RSS :) no matter what a stat() is still more expensive than nothing
Re:RSS mistakes: let's not make them again by jimbobimbo on Wednesday July 17, 2013 @02:58PM (#6059504)
Polling may suck, but it just works. Throw an XML file on your server and you are done, without writing a single line of code.
Re:RSS mistakes: let's not make them again by julien on Wednesday July 17, 2013 @07:15PM (#6060658)
Same as hunting for/picking your own food or sending snail mail to tell things to people. It works, but is not very efficient! Efficiency opens the door for innovation and new behaviors.
Re:RSS mistakes: let's not make them again by ajanuary on Wednesday July 17, 2013 @05:50PM (#6060364)
I've been wondering if feeds would be more discoverable if browsers integrated it with bookmarks. Bookmarking a page with an RSS feed becomes a 'live bookmark' that acts like a bookmark folder with unread counts.

But there's probably a billion edge cases that make it too difficult.

Re:RSS mistakes: let's not make them again by opminion on Wednesday July 17, 2013 @05:55PM (#6060383)
Re:RSS mistakes: let's not make them again by chenster on Wednesday July 17, 2013 @02:05PM (#6059161)
RSS specs doesn't change is actually a great benefit to services build around it. Both http://dealbert.net and http://joboyster.com use RSS as the main data source format.
Re:RSS mistakes: let's not make them again by asdf3 on Wednesday July 17, 2013 @01:59PM (#6059125)
We should add query parameters onto RSS requests, and gradually upgrade while remaining backwards compatible. Create clients and server features that are compelling enough, and people will update their server code.
Re:RSS mistakes: let's not make them again by ancarda on Wednesday July 17, 2013 @02:46PM (#6059437)
What do you mean? /feed?version=1

That's a bad idea if you cache RSS in an xml file, i.e /rss.xml. It would make it more complex if you had to handle query parameters. The parser should just check for the version attribute on the rss node; <rss version="2.0">. Or, even better, it could just check for the availability of server nodes such as "<cloud>".

Or did you mean something completely different?

Re:RSS mistakes: let's not make them again by jerf on Wednesday July 17, 2013 @03:24PM (#6059661)
RSS is already extensible; the eXtensible in "XML" actually works as designed for formats like RSS. You'd have to name a specific feature you needed to actively replace for this to be justified. (You may be able to, but it's not as easy as just handwaving.)
Re:RSS mistakes: let's not make them again by ancarda on Wednesday July 17, 2013 @02:35PM (#6059358)
SubToMe doesn't load for me. I just see a blank page and the subscribe button doesn't work.
Re:RSS mistakes: let's not make them again by julien on Wednesday July 17, 2013 @04:09PM (#6059959)
Weird. Any JS error?
Re:RSS mistakes: let's not make them again by bmelton on Wednesday July 17, 2013 @03:34PM (#6059734)
Offtopic somewhat, but if you're building a feed reader in Python, you can use feedfinder.py[1], to allow for feed discovery. The last feed reader I wrote, years ago, utilized it to great effect.

Every new feed reader on the block shocks me when I have to enter the exact URL to the feed (e.g., domain/xml/feed.xml, or what have you).

[1] - http://www.aaronsw.com/2002/feedfinder/