Speaking in this case isn't simply speaking words out loud intelligibly, but the entire process of thinking and communicating ideas in a foreign language. There is almost always a gap in your knowledge with regards to what you can understand being said and what you can actually say which is why I say it is harder, even if you can say everything that you can say fluently.