Re:Gone in Six Characters: Short URLs Considered Harmful for...
by ikeboy on Friday April 15, 2016 @11:02AM
(#11504860)
Doesn't excluding them in robots.txt solve the search engine problem?
[ Parent
| on HN ]
Re:Gone in Six Characters: Short URLs Considered Harmful for...
by _Understated_ on Friday April 15, 2016 @11:22AM
(#11505032)
Following the rules laid out in a robots.txt file is optional. The reputable search engines tend to play by the rules but dodgy ones? Not so sure.
Re:Gone in Six Characters: Short URLs Considered Harmful for...
by ikeboy on Friday April 15, 2016 @11:44AM
(#11505202)
The dodgy ones don't have access to the URLs. The example was Skype links: if Microsoft scrapes those, they'll follow the rules and not make them available in searches.
If you give your link to a dodgy search engine, you've lost.