This is an incredibly insightful point.
For starters, there is hardly any bijective mapping between words. Translating 'run' to German could mean any of 'rennen, renne, rennst, rennt, renn, laufen, laeuft' and I am sure I am missing quite a bit.
And that only scratches the surface, let's not get started about Grammar or idioms, etc.
The word order really only adds negligible overhead.
Philosophers deny linguistic determinism(a notion that language sets the ceiling for thoughts) but that’s just talking Turing Completeness.
Conversely, you'll only use 'ballon' when you take about un ballon. I hope that's frequently enough for you to remember the word & its gender. At least it looks like the word 'ball', something you don't get with Vietnamese's 'bóng' or Chinese's '球[1]'
And you'll still get funny looks if you say you're getting ready to go to the New Year's ballon, because substitutions fail with homonyms in both directions. And you'll have to learn and relearn about ten thousand of these substitutions.
[1]maybe?