What does testing higher than one's ability actually mean?

I'm guessing either you think the test is an imperfect proxy for a different (more important) skill, or otherwise you have imposter syndrome.

Not OP but I am an abnormally good "tester." It was always easy for me to see what the questions were asking for and could make a go of anything multiple choice. I'm also good at writing essays. It does not make me smart, it makes my abilities especially suited to standardized tests.

Being able to infer information fast, being able to write essays well, etc, are signs of good intelligence and high cognitive capability.

There's no abnormality here i can see. Your score reflects your real ability.

If you cannot solve the same problems given an open ended question, is it real ability? There is definitely something there, but I’m not sure I could be proud of it.

In my case it means performing well under stress, having good time management, and skim reading quickly/effectively. All of these are real things that do matter, but they dramatically boost test scores.

These are skills I use every day as a surgeon. Also grew up extremely poor. Close to perfect SAT/ACT which led to a full ride to good private school. Math degree. NSF fellowship. And, so on..

Would not have happened without the SAT as I was working after school through middle and high school and had very limited 'extracurriculars'

Funny, I rely on those same skills in management all the time. So much so that I wish that the value of those skills, independent of educational subject matter, had been emphasized to me as a young person.

Maybe some impostor, but at the time, there was a penalty for guessing. Between my risk-taking personality and the sense that I wasn't going to get out of the minor leagues by watching pitches go by, I 'swung' at every question and apparently got a lot of questions right that I couldn't actually explain why the answer was correct.

Arguably, someone who knew more than I did but was too hesitant to go out on a limb about it should still have gotten a higher score if the test is designed to assess knowledge.

If I remember correctly, in that period it was still EV-positive to guess if you could eliminate at least one incorrect possibility. Not everybody knew this, though. Simply having someone tell you that (or having the patience to read and understand the scoring rules) is a significant advantage over folks who didn't know.

If your intuition allowed you to guess with a high enough success rate that it overcame the penalty then it is that intuition that gave you an advantage. You didn't just get lucky.

Being able to choose the correct answer in a test environment by process of elimination or similar methodology is a useful skill to have in some non-test circumstances, but it’s not a great test of domain knowledge.

Anecdote: I recently did some tests where I read a bunch of material before hand and used a combination of short term memory and multiple choice trickery to pass with flying colors, but I still don’t have any real knowledge of the subject area. I barely remember any of it, other than a few bits of jargon.

True, but I think the claim is also that said intiuition does not manifest in a general way when dealing with the subject matter; only on the tests. In which case it would not correlate to performance on real world outcomes.

Yes, even the most difficult standardized tests are full of artifacts just based on the fact that there is a correct answer and the test makers have a relatively narrow range of difficulty to work within. In the real world the majority of problems are either trivial or impossible and its not always obvious until you dig in which it is.

There is no exam or test for work-ethic, or (in my case) an inability to refrain from constant yak-shaving.

I will say I'm of decent peer-inteligence (and the test scores put me in the top 1% or higher) but that doesn't account for other aspects of "ability".

I feel my grades more accurately portrayed me as a whole but the test was accurate as to potential if that makes sense.

Standardized tests are extremely structured, and therefore easy to train for, so you can do well on such tests and yet lack other basic abilities which are important for less structured contexts.

Test taking is a skill, and some people are much better than others. Good test takers understand how test takers make questions, and can use context clues and things we know about how answers are worded to get a higher score than if we were just answering the questions based on how well we know the subject itself.

Multiple choice test are a very flawed proxy for knowledge processing ability.

Has anyone ever said "tests fucked me over"? Yes, some people say that, but every time I meet one I see why they tested low. Being able to stay calm under pressure and complete a task on time, and preparing for that, are important life skills, on top of being technically capable.

Tests generate a lot of false positives, but they give false negatives less. If you fail you fail for a valid reason.