As many commenters have pointed out, no one would have a problem had Microsoft trained Copilot on the Windows source code. The fact that they intentionally left it out of the training set is a huge red flag.
Now let me flip that question around on you: What benefit would society gain from that forcing AI developers to do all that extra work?
That sounds like a downside to me, not a benefit. You're basically arguing Copilot, Stable Diffusion, GPT-3, etc (which were all trained on copyrighted works) not existing would be a better situation than them existing. I'm just not seeing that.
Granted, people are not necessarily rational actors, so maybe you could argue it still makes sense to have some protections in place to assuage people's irrational fears. Maybe like some kind of robots.txt for determining whether a page can be used in an AI dataset could serve that purpose. I'd be hesitant to support anything more burdensome than that.
Some other uses are allowed without attribution. Someone can read and learn from open source software without needing to put an attribution anywhere. You could run an analysis of the code on GitHub to find out what percent of code is written in C++. You wouldn't need to attribute every project on GitHub.
Now the debate is whether this applies to training ML models.
These AI systems are highly novel, transformative, and useful. Their development is exactly the sort of thing copyright law was originally created to encourage. If it's hindering them instead, that's a problem.
(And no, I'm not saying people should be allowed to use AI to intentionally launder stolen code; use some common sense here.)