An empire can't retain hostile and unpacified territory beyond a certain limit - which may be set by time, communications, access to local resources and imported logistics, other factors, and all of the above.

If the territory is fully pacified with no significant resistance and/or it's run by dedicated loyalists with plentiful local resources the limit doesn't apply.

Basically an empire can only retain territory against an active threat or resistance within certain limits. If there is no active threat, or the threat is too minor to be a concern, the territory can be considered stable and fully colonised.