It's the other way which is in question, "if you can answer questions, does that mean you definitely have expertise?". Is being able to answer questions the same as having expertise? Are there people who CAN answer questions but have no expertise? If so, how many?
Referencing your other pilot comment, is being able to say what action you should take in an emergency, the same thing as having the calmness and presence of mind and judgement to actually take that action in an emergency instead of locking up and forgetting and panicking? Certainly the pilot who does act well in an emergency could tell you about it, but can all the people who can tell you about it, do it?
Putting it in terms of logic, (A implies B) does not mean that ((not A) implies (not B)).
I passed German at school, despite being unable to speak German. Anybody who spoke German could have tried to have a conversation with me and see that I couldn't, that would be a good way to judge my supposed skills. Any test which is not that, any test which is a proxy to that, or a way to judge that with less effort, is worse than doing that. Teaching me to answer the questions they were going to ask on German grammar, or recite the sentences they were going to expect me to recite, is even worse.
Of course people with absolutely no skill fail tests more than random, who would question that.
That 5% of tests are rigorous and accurately identify squares is not really something I question, but I might question that the people who pass them were "taught to take the test" and nothing else.
(Would you say 7 years of degree, medical school, hospital residency, rotation around departments, counts as "teaching to the test", i.e. "an unhealthy focus on excessive repetition of simple, isolated skills ("drill and kill") [which] limits the teacher's ability to foster a holistic understanding of the subject matter."? - from https://en.wikipedia.org/wiki/Teaching_to_the_test )