Detecting PhantomJS-based visitors

Re:Detecting PhantomJS-based visitors by joshstrange on Friday January 16, 2015 @03:42PM (#8901556)

With the norm nowadays using JS frameworks on the front-end I would expect more and more search engines to use things like PhantomJS to "scrape" the page. If you decide to reject traffic like this isn't there a good chance your SEO will fall?

[ Parent | on HN ]

Re:Detecting PhantomJS-based visitors by nkozyra on Friday January 16, 2015 @03:49PM (#8901607)

I doubt any serious search engine is literally using PhantomJS, so I wouldn't see this as a huge problem.

Google is in all likelihood using V8 directly and is explicit about who it is. In fact, most legitimate search engine spiders are.

[ Parent | on HN ]

Re:Detecting PhantomJS-based visitors by joshstrange on Friday January 16, 2015 @04:56PM (#8901989)

I agree, I was speaking more on trying to block crawlers that look like browsers in general, sorry that wasn't more clear.

[ Parent | on HN ]

Re:Detecting PhantomJS-based visitors by jsoverson on Friday January 16, 2015 @03:48PM (#8901593)

Things like phantomjs, not phantomjs. PhantomJS is often used to automate sites against the TOS.

[ Parent | on HN ]