This is true for me. I can go from notation to my fingers, faster than I can go from notation to singing a tune. It's a hyper specialized skill. I can sight-read treble clef on flute, and bass clef on the double bass, but it's hard for me to sight-read treble clef music on the double bass, even though the transposition of a couple octaves is no problem at all.
When you read these sentences, you can very easily speak them out loud, you don't have to mouth each and every vowel to hear it in your head. This is not the case for music notation, even for very experienced musicians. The clue is that they associate notes not with sounds or intervals, but with absolute positions (fingerings, etc.) on their instrument. Maybe they can sight-read a melody with a little effort, but for most of them it will be much simpler to just play it - and this goes triple if it's not tonal and idiomatic for a style you know. Music notation is really a somewhat instrument-independent tabulature.
Solfege or numeric notation systems, which exist in many versions, are much better for reading in your head once you learn them. They're terrific for singers. But for instrumentalists, usually your instrument is already biased to certain keys, and going from notation->sound in your head->fingers would therefore be much harder than going straight from notation->fingers.