I'm an amateur student of language acquisition. I can only add to this, because I think you're spot on. I knew a few people who landed here on their own and within a few years had no appreciable accent. ALL of them did this by living away from people who spoke their own language. Just too easy to slip into it when you've had a long day cleaning houses or working in an Amazon warehouse.

EDIT: Hilarious typos in a post about language acquisition