If these guys are serious about their business, then they should behave like a good netizen and obey robots.txt directives.