You're going to have to break down the metaphor more explicitly, because I don't think the structure (logical or otherwise) of the poem really points to such a straightforward interpretation.

Like, what is the isle then? Yourself?