News for hackers. Stuff that matters.
Re:Google's fact-checking bots build vast knowledge bank by dm2 on Friday August 22, 2014 @01:17PM (#8212980)
You can ban GoogleBot easily, just put a line in your robots.txt file, but then people won't be able to easily find your site using Google services.

If you provide value to Google they will make an API to allow accessing that data easier.

By scraping do you mean scraping their search results? They offer this, which is nice: https://developers.google.com/custom-search/

Many large sites don't allow scraping because of unnecessary server load (denial of service sometimes) so they'll offer an API where you can download content in a controlled (and monitorable) manner.

Re:Google's fact-checking bots build vast knowledge bank by frik on Friday August 22, 2014 @01:34PM (#8213070)
We need a robots.txt and "noindex" metatag standard for emails.

If one sends an email to an GMail/Outlook.com/Yahoo email address, one should be able to opt-out of their email crawler, advertisement analysis, artificial intelligence analysis, etc.

Re:Google's fact-checking bots build vast knowledge bank by dm2 on Friday August 22, 2014 @01:46PM (#8213125)
The user who receives it can: https://support.google.com/ads/answer/2662922?hl=en

I don't think they do too much storing of email details, they know that it's a sensitive area and that an employee will eventually blow the whistle and it will hurt user trust, which is a big part of their business.

Re:Google's fact-checking bots build vast knowledge bank by frik on Friday August 22, 2014 @01:59PM (#8213181)
I meant it the other direction.

1) Alice sends an email to Bob (GMail user).

2) Bob receives the email, meanwhile Google scrapes the content and extracts its meaning to show Bob some ads and use the facts to improve Google's A.I.

Alice wants a way to mark her email content as "no index".

So that email service provider don't crawl through the content. Exactly like the robots.txt for domains or the "noindex" metatag in HTML head element!

Re:Google's fact-checking bots build vast knowledge bank by lern_too_spel on Friday August 22, 2014 @06:45PM (#8214285)
Why should Alice get to restrict what Bob can do with his inbox?
Re:Google's fact-checking bots build vast knowledge bank by frik on Saturday August 23, 2014 @01:38AM (#8215108)
It's not about Bob, he can do what he want.

It's about the email service provider, that should stop analyze the email text to extract its meaning. Gmail uses it to display ads to Bob, builds a shadow profile for Alice (like Facebook) and trains an artificial intelligence (see headline link).

Re:Google's fact-checking bots build vast knowledge bank by rstupek on Friday August 22, 2014 @05:19PM (#8214018)
It would be nice if there were a way to provide google, bing, et. al. the data so they were constantly crawling sites. A non-insignificant amount of our bandwidth is used up daily by search engines spidering sites.