Because LLM companies have historically been extremely disingenuous when it comes to crawling these sites.

Also because there is a difference between a user hitting f5 a couple times and a crawler doing a couple hundred requests.

Also because ultimately, by intermediating the request, llm companies rob website owners of a business model. A newspaper may be fine letting adblockers see their article, in hopes that they may eventually subscribe. When a LLM crawls the info and displays it with much less visibility for the source, that hope may not hold.