News for hackers. Stuff that matters.
Re:Deep dive into finding RSS feeds by AndrewStephens on Friday December 06, 2024 @04:06PM (#42344448)
Even better, for a few months the browsers themselves would highlight RSS feeds and allow you subscribe right in the browser. It was too good to last.

RSS is great but it has one great flaw in that it doesn't scale that well by itself. If 2 million people subscribe to your feed and try to update it once an hour, that is 48 million requests a day just for RSS.

What does work well (and how things have evolved) is to have a service that polls RSS on behalf of its users. This was the beauty of Google Reader but plenty of replacements exist.

Re:Deep dive into finding RSS feeds by rodary on Friday December 06, 2024 @04:10PM (#42344482)
> Even better, for a few months the browsers themselves would highlight RSS feeds and allow you subscribe right in the browser. It was too good to last.

Vivaldi browser still does that.

Re:Deep dive into finding RSS feeds by qw on Friday December 06, 2024 @06:24PM (#42345695)
> RSS is great but it has one great flaw in that it doesn't scale that well by itself

The readers often cache the result for all users of the same feed. The only exception are readers that are running 100% locally. In that case using Etag or Last-Modified will make each request cheap and manageble.

Re:Deep dive into finding RSS feeds by AndrewStephens on Friday December 06, 2024 @08:10PM (#42346337)
That is my point. RSS by itself is a scaling problem but online readers make it manageable by polling sites on a reasonable rate and redistributing new articles to their users.
Re:Deep dive into finding RSS feeds by lapcat on Friday December 06, 2024 @07:31PM (#42346121)
> RSS is great but it has one great flaw in that it doesn't scale that well by itself. If 2 million people subscribe to your feed and try to update it once an hour, that is 48 million requests a day just for RSS.

Who's updating their RSS feeds once an hour 24 hours a day?

Anyway, a very popular site with that many millions of visitors already has to handle extreme traffic, regardless of RSS.

Re:Deep dive into finding RSS feeds by AndrewStephens on Friday December 06, 2024 @08:08PM (#42346326)
According to my logs, some feed readers poll every 10 minutes or so.
Re:Deep dive into finding RSS feeds by ttyprintk on Friday December 06, 2024 @08:06PM (#42346309)
Surely this is solved by putting RSS behind Cloudflare.
Re:Deep dive into finding RSS feeds by lapcat on Friday December 06, 2024 @08:20PM (#42346390)
Sure, it's solved by Cloudflare breaking the RSS feed entirely, which often happens when people put their websites behind Cloudflare.
Re:Deep dive into finding RSS feeds by compootr on Saturday December 07, 2024 @02:12AM (#42347875)
I think it should be noted that this happens when not configuring things right.

By default, security rules are on, but I can disable security rules programmatically for a hostname too.

fwiw, I once got a ddos on a host running their pages product, and got no charge for it. it also stayed up and didn't give captcha pages

Re:Deep dive into finding RSS feeds by doubled112 on Friday December 06, 2024 @08:49PM (#42346557)
I think Miniflux polls every hour by default. Not sure about others.
Re:Deep dive into finding RSS feeds by xtiansimon on Saturday December 07, 2024 @10:41AM (#42350453)
“Who's updating their RSS feeds once an hour 24 hours a day?”

RSS is a pull-type system, no?. So the end-user is causing the overload problem by hitting the publisher’s RSS feed every hour? The problem is out of the hands of the publisher…

Re:Deep dive into finding RSS feeds by lapcat on Saturday December 07, 2024 @11:27AM (#42350783)
> RSS is a pull-type system, no?

Yes.

The web in general is also a pull-type system.

> So the end-user is causing the overload problem by hitting the publisher’s RSS feed every hour?

There is no overload problem.

And again, this math is off, because an end user is not even awake 24 hours a day: "If 2 million people subscribe to your feed and try to update it once an hour, that is 48 million requests a day just for RSS."

Re:Deep dive into finding RSS feeds by doubled112 on Saturday December 07, 2024 @11:40AM (#42350891)
My home server is awake 24 hours a day.

Is that wasteful? Does that make me a poor Internet citizen?

I'd never considered using RSS feeds from an app and pulling it directly. I have too many devices for that to be a great workflow.

Normally I'd say I was an outlier, but my gut feeling on this is that the people still using RSS and people self-hosting would overlap in a big way.

Re:Deep dive into finding RSS feeds by lapcat on Saturday December 07, 2024 @01:27PM (#42351682)
> Is that wasteful? Does that make me a poor Internet citizen?

I personally don't think it's a problem. As I said, "a very popular site with that many millions of visitors already has to handle extreme traffic, regardless of RSS."

> I'd never considered using RSS feeds from an app and pulling it directly. I have too many devices for that to be a great workflow.

A lot of apps have sync to handle that workflow.

> my gut feeling on this is that the people still using RSS and people self-hosting would overlap in a big way.

My gut feeling is that most RSS users, including myself, simply use client apps.

Re:Deep dive into finding RSS feeds by xtiansimon on Sunday December 08, 2024 @12:33PM (#42358505)
> "...an end user is not even awake 24 hours a day..."

Sorry, I wasn't thinking about the end user literally pushing a button. I was reading into the discussion here what I imagined to be a gap in the overall design philosophy of RSS where the end-user, using their own app or preferred methods, poll all of their favorite websites selecting among common defaults (Select the frequency you like to poll this RSS feed: weekly? daily? hourly? hehe minutely? secondly?).

In this scenario the user doesn't need to be awake 24 hours a day. They just need to use software which stupidly hits someone's server like they're doing a denial of service attack. As your comment alludes no one needs to be that up-to-date.

In contrast, there was another solution: "...service that polls RSS on behalf of its users."

Again, reading the discussion I imagined the wild west of the internet has inexpert users choosing unsane defaults and overloading small self-hosting content providers. Compared to third-party aggregators who are trying to make money by solving a "Tragedy of the Commons" problem.

Re:Deep dive into finding RSS feeds by lapcat on Sunday December 08, 2024 @12:42PM (#42358580)
> In this scenario the user doesn't need to be awake 24 hours a day. They just need to use software which naively hits someone's server like they're doing a denial of service attack.

But the user's computer would need to be awake 24 hours a day, which is usually not the case.

And once per hour is not a DoS attack.

Re:Deep dive into finding RSS feeds by cknight on Friday December 06, 2024 @08:06PM (#42346308)
> Even better, for a few months the browsers themselves would highlight RSS feeds and allow you subscribe right in the browser. It was too good to last.

I use the Awesome RSS add-on for that in Firefox: https://addons.mozilla.org/en-US/firefox/addon/awesome-rss/

Re:Deep dive into finding RSS feeds by sitkack on Saturday December 07, 2024 @07:40AM (#42349391)
Firefox removing RSS support was their capitulation in supporting the open web.
Re:Deep dive into finding RSS feeds by monooso on Saturday December 07, 2024 @08:08AM (#42349557)
That is very ambiguous phrasing.
Re:Deep dive into finding RSS feeds by sitkack on Saturday December 07, 2024 @01:36PM (#42351764)
Firefox removing RSS support was the nail in the coffin for their support of the Open Web.

How long after Google Reader was canceled did FF remove RSS?

Re:Deep dive into finding RSS feeds by aeonik on Sunday December 08, 2024 @08:34AM (#42357033)
Google shut down Google Reader on July 1, 2013.

Firefox removed RSS support in version 64 on December 11, 2018, approximately 5 years and 5 months later.

Re:Deep dive into finding RSS feeds by sitkack on Sunday December 08, 2024 @10:16AM (#42357601)
Thank you! I would have never looked that up.
Re:Deep dive into finding RSS feeds by ndriscoll on Friday December 06, 2024 @11:11PM (#42347207)
A $160 n100 minipc should have no problem serving an essentially static file out of nginx at well above that rate. If you use proper caching headers and don't publish new items that often, it should basically be idle.
Re:Deep dive into finding RSS feeds by DanAtC on Saturday December 07, 2024 @01:35PM (#42351751)
A lot of clients don't respect caching headers[0], but yeah, static content hosting is a solved problem.

And there are always CDNs if you really have a large, global audience.

[0] http://rachelbythebay.com/w/2024/10/25/fs/

Re:Deep dive into finding RSS feeds by jrm4 on Saturday December 07, 2024 @10:18AM (#42350316)
But that's just a Cloudflare problem, no?
Re:Deep dive into finding RSS feeds by rakoo on Saturday December 07, 2024 @07:42PM (#42354067)
That's exactly why pubsubhubbub, now websub, has been created. Around 10 years ago.