Re:How we learnt to stop worrying and love web scraping
by Aeolun on Wednesday September 09, 2020 @11:41AM
(#24422210)
I think this depends entirely on what you are indexing. From my experience with some 100 ish scrapers for news sites a few would break literally every day.
And the only thing we really wanted was article title and date.
And the only thing we really wanted was article title and date.