I assure you that there are less than 10k unique onion addresses. This is a huge overkill to have a distributed system to crawl something this small.

edit: onion services, not addresses