If it was uniquely an historical question then another text file to handle AI requests would exist by now, e.g. ai-bots.txt, but it hasn't and likely never will, they don't want to even have to pretend to comply with creator requests about forbidding (or not) the usage of their sites.