Yes. Some of the oversea territory had a weird status in the past (see openplatypus comment), but nowadays they are almost all DROM which mean, it is part of France, and being born there makes you French (and makes you a EU citizen).