Whatever difficulties this view presents, this is the only view that makes sense.
It's like saying you can modify and audit Microsoft Windows or other proprietary software because you can write and read the bytes of its object code.
Modification: There have been repeatedly successful case of transfer learning where you take a pretrained deep neural network and train on a new dataset (even very small).
Vulnerability: you dont need the training dataset to test vulnerabilities, you just either very specific input (that you probably gonna have to create yourself because they are so specific that they are not even in the dataset) or using one input that you corrupt for attacking. And as far as I know it doesn't sound harder than making test for regular algorithms.
To me, this is the crux of the matter. If the view is taken that software distributed under a "free-software" compatible license is non-free without the ability to obtain all training sets for any models, there's going to be huge difficulty in incorporating ML into free software in many cases. Since many datasets aren't able to be redistributed freely (e.g. licensing, legal, cost), that's an enormous advantage for non-free offerings.
A possible route around this might be 'community curated' datasets, where contributors freely license their data in the same way as we do code. It'd be interesting to see if they come with an analog of the AGPL - ie. models trained on this dataset must be released to the user as source. (This might already exist?)