I think all French territories are part of the EU, but some other countries work differently. The prime example is St. Martin, where the French part is in the EU, but the Dutch isn't (yes that technically puts a EU border through the middle of the island, although there were no pass controls when I was last there.).