News for hackers. Stuff that matters.
Re:Amazon Web Services: Clouded by Duplicate Content by bdb on Sunday June 27, 2010 @05:30PM (#1466273)
Sorry, but the author is disqualified by this sentence:

"Now there were no external links to these AWS subdomains but, being a domain registrar, Google was notified of the new DNS entries and went ahead and indexed loads of pages."

Re:Amazon Web Services: Clouded by Duplicate Content by gxti on Sunday June 27, 2010 @06:45PM (#1466430)
To clarify:

* Google isn't a domain registrar. If they were then one would hope that google.com would be registered under them, not markmonitor.com (a popular corporate registrar)

* The server hostname is a subdomain of amazonaws.com and doesn't require registration.

* In fact, the entire name is autogenerated from the IP address using a simple substitution rule.

* I'm not sure how Google finds these, because their own "what links here" doesn't turn anything up.

* Side note: while searching for backlinks in Google, I noticed that this article seems to have been copypasta'd to a dozen other blogs. That's SEO, baby!

EDIT: Damnit HN, you look like reddit, why can't you use a sane text preprocessor like reddit?

Re:Amazon Web Services: Clouded by Duplicate Content by jonknee on Sunday June 27, 2010 @06:51PM (#1466446)
Re:Amazon Web Services: Clouded by Duplicate Content by gxti on Monday June 28, 2010 @09:02AM (#1467777)
Thanks for correcting me, I've never used google apps so I'd never noticed you could register domains there. Personally I'd still register my own domains elsewhere as I've had bad experiences with combined registration and hosting, and even the almighty Google has been known to partake in shenanigans.

In any case I'm pretty sure you don't even have to be a registrar to get lists of newly registered domains, not that it matters because the rest of my post addresses why registration has nothing to do with this.

Re:Amazon Web Services: Clouded by Duplicate Content by thehodge on Sunday June 27, 2010 @08:03PM (#1466567)
* Google isn't a domain registrar. If they were then one would hope that google.com would be registered under them, not markmonitor.com (a popular corporate registrar) As pointed out, Google is a domain registrar - http://www.theregister.co.uk/2005/02/01/google_domain_seller...

The server hostname is a subdomain of amazonaws.com and doesn't require registration. * In fact, the entire name is autogenerated from the IP address using a simple substitution rule.

I don't have technical answers to these

* I'm not sure how Google finds these, because their own "what links here" doesn't turn anything up. Google has a tonne of ways we don't know about for finding domains, there is at least one case I've seen where a deep URL has been indexed after only a mention in gmail

* Side note: while searching for backlinks in Google, I noticed that this article seems to have been copypasta'd to a dozen other blogs. That's SEO, baby!

Thats actually content scraping and aggregation, SEOMoz has a tendancy to get there articles ripped off with much haste and from 150+ blogs (I have been mentioned in them a few times and they always set my Google Alerts wacky)

Re:Amazon Web Services: Clouded by Duplicate Content by jluxenberg on Sunday June 27, 2010 @06:33PM (#1466401)
Which brings up a good point: how _does_ google find these domains? Are they just blindly scanning the entire IP block?
Re:Amazon Web Services: Clouded by Duplicate Content by bkrausz on Sunday June 27, 2010 @06:48PM (#1466438)
It's possible another user of that DNS name (they do get reused) linked to it at some point and Google picked it up.