Maybe it depends on version or something? I've had the same config for years though over several very different OSes and presumably versions of everything.
I thought so as well, so the reply to this message surprised. I looked into it, and the following is the case on my Debian 18.04 machine running KDE:
With specifying the key through `IdentityFile` I can happily connect without a running ssh-agent. So it's true, that it can do without using the ssh-agent. But if the key has a password, it will prompt for it everytime it's used. I wouldn't phrase it "the agent isn't used at all", though, because for when ssh-agent is running, and it contains the key+password, it seems to happily use that agent, as it doesn't prompt for the password anymore.
Side note: If my understanding is correct, `IdentifyFile` lessens the need for consulting the ssh-agent as stated above, but, except from consulting it for the key password, the agent might be consulted for one more reason as well: Iterating over keys if using the specified one proved unfruitful. For this to stop, you'd have to specify `IdentitiesOnly yes` as well. But this I didn't test, so it's based on theoretical understanding only.
Edit: Oh, the last part was already explained in some other thread which branched of from this. So this post didn't actually provide some new insights, it seems.
If you specify -i or IdentityFile, the agent isn't used at all. You will have to type the keyfile password even if the key is in the agent, therefore making the agent useless.
This "feature" has been a big annoyance for me, since I like to use different keys per machine.