A compilation is protected only as a completed work. You can't, for example, photocopy the yellow pages and resell it. But you could copy each and every phone number from the yellow pages and use those numbers in your competing directory.
One additional point that the book makes is that one should keep in mind is your robot showing up in the logs of places you search. You don't want to overload sites, or get on the wrong section of the site admin's radar.
So, facts are not copyrighted, but the way facts are presented are copyrighted.
I still find it hard to believe that what this person had been doing is legal. What is the difference between the scraping he does and copying the phone directory and selling it on. In fact that is not a fair comparision since the wikepidia article suggests that the fone directory was legaly obliged to list these businesses while an individual or google and yahoo are not legaly obliged to enter every business in their directory. In this regard then the choice of information that these directories have made makes their directories copyrighted perhaps.