Re:Ask HN: Has Google/Bing won the search engine war?
by palakchokshi on Monday March 31, 2014 @02:42PM
(#7503314)
So a "legit" crawler has no recourse but to resort to implementing technology that'll circumvent anti-crawler code? Can't say I didn't try to go legit :)
[ Parent
| on HN ]
Re:Ask HN: Has Google/Bing won the search engine war?
by timr on Tuesday April 01, 2014 @03:40AM
(#7506688)
Well, no: you ask, and if they say no, you don't get to crawl. The rules don't change because you think they're unjust.
If they catch you crawling, they'll send you a cease and desist letter, so do it at your own risk.
Re:Ask HN: Has Google/Bing won the search engine war?
by palakchokshi on Tuesday April 01, 2014 @12:22PM
(#7509446)
Agreed. I would rather not go down a route of building my product and then have a bunch of cease and desist orders taking away a lot of my search results. Asking nicely is the next step for me.