News for hackers. Stuff that matters.
Re:Ask HN: Can web scraping be the basis of a viable busines... by xedarius on Saturday December 18, 2021 @12:21PM (#29606307)
I went for an interview once at a hedge fund. There were a surprising amount of questions about web scraping. I very much got the feeling it was an active and ongoing problem. So yes I do think there’s a business in there.
Re:Ask HN: Can web scraping be the basis of a viable busines... by lubujackson on Saturday December 18, 2021 @02:19PM (#29607628)
AFAIK Plaid does a fair bit of old school scraping behind the scenes - a lot of the "nice new web" is built on the backs of old kludgy websites doing the same scraping things people were doing 20 years ago.

Finance/banks are especially... inconsistent, to say the least.

Re:Ask HN: Can web scraping be the basis of a viable busines... by fforflo on Saturday December 18, 2021 @01:00PM (#29606735)
Having recently worked for a hedge fund I noticed that too - although my idea predated that engagement . That's probably my first go-to market (if anyone has leads send me end email) .

Hedge funds actually call this "alternative data".

Re:Ask HN: Can web scraping be the basis of a viable busines... by dreyfan on Saturday December 18, 2021 @01:42PM (#29607227)
If you worked for a hedge fund and you know they purchase scraped data from quite a number of vendors then why are you asking if it’s a viable business model?

Of course it is.