Perhaps "scientist" is too vague a term. I mean people with a "in their bones" appreciation of experimentalism, empirical adequecy etc.
What you're saying overall aligns with my view -- no on here is defining empirical adequacy on ML systems as "theories of natural language". As soon as you do this, which would be step1 in any experimental endeavour, the whole show collapses.
No one here is being taught to target empirical systems and develop criteria of empirical adequacy... thinking somehow "accuracy on benchmarks" tracks "ability to understand language" -- this, as we both think, is mad.
I guess my cultural issue is better described then as "pureish mathematicians" having come to own an empirical field, with all the predicatable nonesenses which follow (whether they are discrete or continous).
>> (...) the older generation of csci academics and much more aligned with my view).
That's true and hardbreaking. I don't know what we can do about that. The success of big tech corporations has broken computer science education, I'm afraid, and now all that graduates care about is how to make a six-figure salary at a FAANG.
>> All you see is numbers and functions, and all I see is problems being solved by these AI/ML methods and often in quantifiably better ways than intelligent humans can.
Maybe in playing chess and go, but in any other task that neural networks have been compared to humans directly, the ability to outperform humans is more informative of the limitations of the measures of performance, than of the capabilities of the tested systems (typically, deep neural networks).
Take large language models (LLMs) for example. Last time I looked -it was a while ago but- LLMs like BERT, ERNIE and their famous friends scored very highly and even above humans in natural language understanding tasks, such as those in the GLUE and SUPERGLUE benchmarks.
You'd think that a system that scores 85% in a language understanding task has developed at least _some_ ability to understand language. And yet this is not the case. LLMs, despite their ability to score highly in formal benchmarks, remain dumb as bricks and as incapable of understanding human language as a rock.
We know this because every once in a while someone in the field of NLP looks away from their favourite leaderboard for long enough to ask serious questions about the performance of systems, as for example [1] who started by asking why a language model like BERT would be any good at argument comprehension (BERT scored only three points below humans in a relevant benchmark task). Everytime anyone undertakes this kind of analysis of the abiltiy of LLMs to understand language, the conclusion is that they don't, and instead they learn "cheap tricks" and overfit to statistical regularities of the benchmark datasets. Similar analyses have found similar results in machine vision (e.g. [2], [3]) etc.
The question then is, in what sense, exactly, is an automated system solving problems "in quantifiably better ways than intelligent humans can", when that system is not even solving the problem it's supposed to be solving, like the problem of understanding language by LLMs? The idea that those systems "solve problems", when they 're really only overfitting to their training sets, is an artifact of the shoddy scholarship that is typical in deep learning research, and the OP is right to be concerned that there is little understanding between machine learning researchers of what, exactly, it is that their systems are doing- or even trying to do.
And, as the OP points out, I'm afraid that really is the result of a poor background, not only in sciences and mathematics, but in AI and even machine learning itself, among the current generation of machine learning researchers. Contrary to what the OP says, though, this paucity of relevant background is the result of an influx to the field of people from the sciences, who have a background in continuous mathematics, but not in AI or computer science. It is, indeed, a background in computer science that can best guard against misconceptions about computers being wonder machines with magickal intelligence abilities. At the very least, you don't have to explain to a computer scientist why solving one instance of a class of problems does not mean a general solution for the entire class. That comes built-in to most who have graduated in the discipline. But is it built-in to phycisists, or biologists, who study very different systems?
________________
[1] Probing Neural Network Comprehension of Natural Language Arguments
https://aclanthology.org/P19-1459/
[2] Intriguing properties of neural networks
https://research.google/pubs/pub42503/
[3] The Elephant in the Room
https://arxiv.org/abs/1808.03305