This has been my experience as well. I'm also am afraid that these researchers will succumb to a similar replication crisis that devastated the social sciences [0]. If you can't necessarily explain why something works, it's probably difficult to explain trying it again in the future didn't work

[0] https://en.m.wikipedia.org/wiki/Replication_crisis

It's more like the opposite problem.

Psychology researchers try to justify deep, compelling theories of human behavior with small experiments that are hard to reproduce.

Deep Learning experiments are comparatively easy to run and reproduce because the field has both a common set of performance benchmarks and a large community of people implementing cutting-edge ideas in common frameworks. The harder part is building useful theory on top of those experiments.

This is not really the case. I have personally quite literally implemented papers the week after they appeared on ArXiv and they worked. Anecdotical yes, and there have been papers that nobody was able to replicate, but at large the community is quite good at recognising which work can be used, and this work is then implemented and disseminated.

The situation is slightly better than social sciences though. Social science experiments, if any, can hardly be repeated. AI experiments can be easily redone if the source code and datasets are released into the public.

I'm thinking that the deep learning field will become like web development: it's becoming so simple that everybody can do it.

This article gives some evidence: [1], quoting:

> researchers were a bit perplexed when actress and director Kristen Stewart [2] appeared as an author on a machine learning paper.

[1] https://techcrunch.com/2017/01/19/kristen-stewart-co-authore...

[2] https://en.wikipedia.org/wiki/Kristen_Stewart

I might be wrong, and I don't want to question the possibility that someone from a non-technical domain can contribute to technical domains (in particular because neural networks and style transfer are basically high school mathematics; and since this happened before [1]), but it seems to me that the paper in question does not contain much technical innovation, but it is rather a demonstration of how existing techniques can be used in practice [2].

[1] https://en.wikipedia.org/wiki/Hedy_Lamarr

[2] https://arxiv.org/abs/1701.04928

I've heard that claim a couple of times now, but never read or heard experts in the field say anything like that. Do you have a source?

Of course, you can do fancy things with neural networks without fully understanding how they work, or why they behave like they do. But my impression is that common applications, like image classification, are quite well-understood now.

I think you are right to make that distinction. Research papers are probably more about other, less known applications.

What? They do explain their results, and as you can see from the references section of any paper, other researchers read the published research and then build their new models using that information and continually refining the process.

Sure, they do their best to give an explanation after discovering it works, but nobody ever publishes the hundred other models they tried that don't work out.

This is a problem in most areas of scientific research. There is no incentive to publish negative results, so no one does.

Yes, they explain it, but in a very hand-wavey way.