Extract article text from RSS feeds around the globe en masse.

Mini guide here - https://www.reddit.com/r/webscraping/comments/fbknv4/extract...

Articles given via RSS feeds usually don't contain ads, right? Might it be possible to use content with no ads from RSS and the same content from the home webpage with ads to build a text extraction model that works beyond RSS?

There are AI solutions that can work on the HTML and manage to avoid the ads. Checkout https://blog.scrapinghub.com/extracting-clean-article-html-w...

That is typically not the case. Some are just a headline or a headline and a short paragraph of text (the most common). Some are the full text without HTML (thus no ad links -- mostly indie blog pages). Some are the actual page.

What is this actually for?

The main API extracts text from news pieces, blogs, PR content, and other text-heavy pages with a GET request.

The free account gives you access to a GUI where you can paste/upload your URLs to extract text from. The "playground" is a collection of RSS feeds with links you can extract text from, to try out the capabilities of the API.

Right I understand what it does, but what is it for? Why would someone want to do this?

Personal use case was compiling a news dataset to train a topic model.