There would be huge benefits to going that route, but they didn’t. That’s what makes me think that it’s impossible because they needed a huge volume of data, and the only way to get enough was to take it without consent of the license owners.
Simple: They disagree with you on the risk of that happening and the potential cost if it did.
If this is not derivative work, how come they didn't use their own codebase instead of taking the opensource ones ? Either their codebase is low quality, either they think it would leak their IP, meaning it's derivative work. Neither solution sounds great for them
Convenience probably.
Using their own code would have required them to coordinate inhouse, and carefully and expensively sanitise everything, because committing their code, which was largely never written to be open sourced, to run the wild Copilot experiment (which it still is at this point) probably seemed a little riskier than they liked, potentially creating some kind of security disaster. So just use the stuff that is already in the open and thus better be secure enough as is, at what they apparently deem the relatively small risk and costs of lawsuits happening.
I could also see them wanting to test the water on how the license thing flies.
I am more than relatively certain that it would provide far worse quality, if not be unusable by developers on average.
How is that obvious? I am relatively certain that MS committing their entire code base as training material would do the trick, if that's what it took. Or, additionally, maybe licensing some other huge high quality code bases (restricted to just training the ai) for a few million bucks? It's not like it would be an issue to find vendors happy to remonetize their already written code.
Given what's at stake here and who sits at the helm, I don't see how Copilot would not or could not be moved forwards regardless.