I don't think so. Neither my native nor adopted tongues are English. My native tongue actually has more syllables than the adopted one, for some lowish numbers. Yet, I still count in my native tongue, e.g., when counting the reps while lifting weights.

I'd say I speak better my adopted tongue than my native one. People from my country say I speak with an accent, while locals are surprised to learn I'm a foreigner, since I speak with no accent. When I think about things, I also use my adopted language (I went to school here, so it's easier, since I've learned the concepts in this language). Hell, I sometimes use English for my internal thinking. Just not when counting or adjacent tasks.