It's sort of like the color perception problem [1]. Dogs and machines do see colors, but what do they see?
1. https://newrepublic.com/article/121843/philosophy-color-perc...
Probably the most cited paper regarding this debate is by Marc Bekoff, "Cognitive Ethology: Slayers, Skeptics, and Proponents" (http://cogprints.org/160/1/199709005.html). Your original comment would be categorized as a "slayer" a position which is widely criticized. In fact Bekoff's focus is on canines and he used your exact example with dogs, but to opposite affect.
I do wonder about the theoretical bird scientist trying to figure out the "fixed action patterns" of other animals. If anthropomorphism is the way to go, surely it goes in the other direction in some way.
I don't see a test that majorly distinguishes it from a human. It appears to be following the same process with a few tweaks around the edges. There are some exceptions in the 2-5 situations in Go where a human can actually use optimised logic to determine what will happen; but they aren't the meat of the game.
I don't recall ever reading in a technical paper, or in an interview, a leader in the field of ANNs claim they were thinking. If you have, I'd like to see a reference. Most are fairly honest about the differences between artificial neurons and real ones, and between human cognition and what ANNs are doing with data.
But ML, AFAIK, is so simple; its literally a glorified polynomial functions. The only thing it get going for it is the large data set that we can train it on. It cannot "learn" anything from a small data set and extract any information out of it without a human imposing his/her knowledge on it.
For instance, take the concept of an even number. This simple knowledge is so powerful in solving algorithmic problems. But, its very hard to make a machine learn of this concept in general.
I think the problem is really overestimating how "intelligent" human are. We are only as intelligent with respect to our imagination. Its possible that there is an entire class of intelligent outside of our imagination that we cannot fully grasp its intelligent. Similarly, I am only conscious with respect to my own consciousness, but there may be another class of consciousness that is unimaginable to this monkey's brain.
I’ve noticed at least as many people under-anthropomorphize as over. People who seem obsessed with human exceptionalism and are personally offended at the idea that plants and animals (and computers!) might have subjective experiences like our own.
But to me it seems obvious we are far more alike “lower” species than we are unlike them. I would say the cases of human exceptionalism are actually extremely rare. The main source of our uniqueness is that we amalgamate other species, not that we have transcended them.
My theory is that we are terrified that we might be simpler than we think, because socially we behave as if we are so singular. If we are simple, and animals and machines are like us, then maybe we should be treating them with more reverence.
But being afraid of that is OK for a random person. For a machine learning researcher I would hope they are more careful about what we have evidence for (the similarities between us) and what we don’t (that there is some ineffable magic about humans).