News for hackers. Stuff that matters.
Re:“This story is being previewed exclusively on Apple News ... by threeseed on Monday September 21, 2015 @01:49AM (#10250580)
It should be fine if they add it to their robots.txt file.

It could alongside the other interesting exclusions: http://www.wired.com/robots.txt

Re:“This story is being previewed exclusively on Apple News ... by codewithcheese on Monday September 21, 2015 @02:14AM (#10250641)
Some interesting disallows there such as Disallow: /topics/Brooklyn_Police_Department. Can anyone comment on why this is?
Re:“This story is being previewed exclusively on Apple News ... by chkuendig on Monday September 21, 2015 @06:45AM (#10251413)
That seems to pop up semi-frequently, It's basically a voluntary "right to forget" provided by the publishers.

http://f.cl.ly/items/1o2k3u2J343O3y3X290J/Image%202014-04-07...

http://harpers.org/robots.txt

https://twitter.com/waxpancake/statuses/453024948475293696

Re:“This story is being previewed exclusively on Apple News ... by codewithcheese on Monday September 21, 2015 @10:59AM (#10252730)
Very interesting, good to know. Thanks!
Re:“This story is being previewed exclusively on Apple News ... by PhasmaFelis on Monday September 21, 2015 @03:41AM (#10250893)
And /topics/Advil? Wha?
Re:“This story is being previewed exclusively on Apple News ... by chinathrow on Monday September 21, 2015 @06:11AM (#10251336)