If an art student was able to gain those abilities, would you then argue they couldn't legally create art?

These are all true. But they apply equally to a search engine index and that has already been found not to violate copyright and to be very useful to society.

> These are all true

That was the point.

Whether or not there is copyright infringement going on (or whether copyright law is an appropriate regulatory framework for ML models), I frequently see claims like "ML training is no different from what humans do" repeated, in spite of its incorrectness.

Notably none of those things, if they did apply to the art student, are copyright violations. The speed, scale, versatility, and ownership of a machine learning model has no bearing on its ability to violate copyright.

Good reasons not to assign copyright to their output.

I don't have strong feelings about that either way, but that's not what this post is about.

The interesting thing to me is determining when something that is acceptable for humans to do at human scale is also acceptable for machines to do at industrial scale.

There are many examples. One is face recognition - clearly it is acceptable for individual humans to do this at small scale, but systematic identification of everyone on a street or in a stadium has different implications for society.

(In this case it almost doesn't matter whether the surveillance is performed by humans or machines - it's the scale and systematic nature that changes the equation.)

This just means students better grow bigger brains!