Re:Scraping Shock: Why Web Data Is Getting Too Expensive to ...
by fidansin on Monday January 26, 2026 @09:00AM
(#46765720)
I just worry we're collapsing two things into one bucket:
harder in absolute terms vs harder relative to how much real engineering effort teams are willing to invest.
Those aren't the same, and to me the distinction matters.
I feel it really comes down to priorities.
Scraping has always been a means to a end for most companies. Get data and then use it for something valuable. Before getting the data was easy, but now it is getting increasingly harder.
I think the key here is highlighting the fact that the time of cheap/easy/low skilled access to web data is ending. Companies either need to skill up on understanding how to bypass anti-bots or pay someone else to do it for them and they focus on the data.