I don’t understand, companies that want to crawl can pay for services like brightdata or crawlbase, the barriers don’t apply to them

This ends up hurting individuals and small companies that are harmless and cannot afford to pay