News for hackers. Stuff that matters.
Re:Pattern - Web Mining Python lib by stevejohnson on Sunday October 14, 2012 @03:02PM (#4652246)
NB: screen-scraping Yelp is against the TOS and you'll get shut down pretty fast if you try it.
Re:Pattern - Web Mining Python lib by Terretta on Sunday October 14, 2012 @03:08PM (#4652269)
Exactly. Screen scrape Google search results instead, I've heard that works great. Bet your business model on it, I've heard. ;-)
Re:Pattern - Web Mining Python lib by bigthingnext on Monday October 15, 2012 @02:14AM (#4653978)
Well, Google "screen scrapes" millions of websites. They bet their business on it and they seem to be doing OK.

If you upload something to an http server connected to the public internet on tcp/80, and you don't exclude the path to it in robots.txt, then should anyone be surprised if it is copied? HTTP clients don't read TOS.

If Google had to read and interpret every every website's TOS, I doubt they could easily, if at all, produce an index the size of the one they have. It seems by ignoring a "fear of scraping" they managed to produce something valuable that the courts seem to side with in spite of offended copyright holders.

Moreover there's no requirement for them to make their "cache" publicly accessible. But they do. And again this has held up in court quite well. I doubt anyone would be surprised that people are using it. Or "scraping" it if you want to play word games.

Re:Pattern - Web Mining Python lib by salimmadjd on Tuesday October 16, 2012 @03:46AM (#4658799)
I see you worked at yelp before so I will not elaborate more, but it's not too hard to scrape yelp if you know what you are doing