My passing familiarity with aerodynamics and control theory suggests that you could derive a multidimentional shell in parameter space (of wind, temperature, airspeed and other conditions), to form an envelope within which the plane will behave predictably, so that you can prove whether lowering flaps by 5 degrees at a particular point will help you land safely. Accounting for model uncertainty, that envelope would likely be tighter than it could be if we knew our physics better, but that's still far better than a black box ML model that doesn't give you guarantees that similar inputs will lead to similar outputs (there's a mathematical formalism to this whose name escapes me now).
You might be thinking of the K-lipschitz-smoothness or K-lipschitz-continuity of a model for a given norm, i.e. $\Vert f(x)-f(y)\Vert \leq K \Vert x-y\Vert$. A guarantee that K is smaller or equal a certain value is called a Lipschitz certificate . We can by now give this type of guarantee in balls around the training data (coming out of Adversarial example research), but with some limitations and generalization of Lipschitz certificates to test data and/or other norms is pretty bad in general.
I personally think that the term "black box ML model" needs to die with respect to neural networks, the theory work being done has pried open that box sufficiently by now that we can start reasoning about them somewhat. People just generally don't like the answers because it unveils limitations or challenges
To put it plainly, if your paper has a "Framework" section that has some text titled "lemma" and "theorem" then you also need some text titled "proof"
OpenAI's focus is producing bleeding edge technology demos and being open source AI platform company. They cut corners just to able to show what brute forcing the state of the art can do just now.
This is not critique. It's just a fact that their focus is different from fundamental research.
Would Neural Tangent Kernels help with this, or are they just as impenetrable as latent spaces?
Ditto for AlphaGo, AlphaZero, and most other AI systems.
If one wants to argue that we shouldn't be doing AI science this way, then that's fine. It's just a different conversation.
GPT-2 (fully weighted) was arguably reproduced as well, eventually 'forcing' OpenAI to release the model they had held back [0].
0: https://www.reddit.com/r/MachineLearning/comments/by4n8p/p_s...
I admit that it's possible the entire world is currently "doing AI wrong," though. T5 and other new projects have been better with respect to replication. But many datasets are still locked behind paywalls, or only accessible if you have an .edu email address.
Personally, I struggle to see what new thing we learned from GPT-2. Did we learn something about the physical world? About how human minds work? About how language works? It's a language model after all. All we learned is that throwing a large dataset to a hard problem can produce results that are difficult to evaluate.
"Science" means "knowledge". If we haven't learned anything new from GPT-2 then it hasn't contributed to science. It's impressive, like a jetliner is impressive, or an aircraft carrier is impressive, but it's not increasing our body of knowledge about the world and ourselves.
For example, can you give a proof of superconvergence? What’s the exact learning rate that causes it, and why? Did you know that you can often get away with a high learning rate for a time, and then divergence happens? What’s the proof of that?
Give a proof that under all circumstances and wind conditions, lowering your airplane’s flaps by 5 degrees will help you land safely.
Also, what about datasets that you’re not allowed to release? I personally despise such datasets, but I found myself in the ironic position of having a 10GB dataset dropped in my lap that was a perfect fit for my current project. Unfortunately it wasn’t until after training was mostly complete that we realized we hadn’t asked whether the author was comfortable releasing it, and indeed the answer was no. So what to do? Just don’t talk about it?
I guess the list is good as a set of ideals to aim for. I just wish some consideration was given that you often can’t meet all of those goals.
Most of OpenAI's work would be excluded by this checklist. I don't think anyone would argue that OpenAI doesn't do important work, and that their results are in some sense reproducible.