Parsing, rendering, and trusting that the payload is consistent from request to request is your problem. You can connect to my server, or not. I really don’t care. What you cannot do is dictate how my server responds to your request.
If I think a given request is from an LLM training module, I don't have any legal obligation whatsoever to return my original content. Or a 400-series response. If I want to intersperse a paragraph from Don Quixote between every second sentence, that's my call.
The client is under no obligation to be truthful in its communications with a server. Spoofing a User-Agent doesn't "dictate" anything. Your server dictates how it responds all on its own when it discriminates against some User-Agents.
Even redistributing secret material you found on an accidentally open S3 bucket, without spoofing UA, could be considered intrusion if it was obvious the material was intended to be secret and you acted with bad intent.
It is also a technical fact that a client can send any header it wants.
It seems like some website owners want to have their cake and eat it too. They want their content indexed by Google and other crawlers in order to drive search traffic but they don't want their content used to train AI models that benefit other companies. At some point they're going to have to make a choice.