When we're speaking of proficiency and comparing them between reading, listening, and speaking, you look at them through a similar lens. How well can you understand or communicate ideas in the language? Can you convey complicated and nuanced thoughts in your target language? Can you understand those? How would your speaking ability compare to a well educated native speaker? How does it compare against a less educated native speaker?

You feel more fluent in speaking because you're only working within your own ability while listening you'll encounter things you don't understand or situations where it's difficult to understand things all the time. That doesn't mean you're more proficient at speaking and in most cases you aren't going to be nearly as proficient at speaking as you think.