I've found that, ironically, Vietnamese that already speak English are also more likely to understand my amateur Vietnamese. Maybe that's because they're more familiar with the sounds of English and can make more educated guesses about what I'm trying to say.

I suppose this is true of all tonal languages but Vietnamese is very unforgiving. Even when people really butcher the pronunciation and grammar of English I can still usually guess what they're trying to say. If you don't really nail the pronunciation and tones of Vietnamese people won't understand you at all. I've seen a lot of foreigners give up in frustration even after years of living here.

If you really want to learn to speak tiếng Việt you have to find a real teacher and drill the basics of tones and pronunciation. If you try to pick it up a phrase at a time chatting with people you'll never get it.