I wouldn't worry too much about it, English is prevalent and useful enough that as long as gets to a reasonable level early enough, he should be able to pick stuff that separate him from your specific manerisms

Tangentially related, I see more and more kids developing a "TV Accent" in Spanish in their early years which eventually gets replaced with their local one I'm guessing based on their exposure to parents and media vs School/Parents/Every day life