I agree with almost everything you said. Fair Use is definitely protected though, as it's a core component of the US copyright law that helped shape the rest of the law, not an exception or amendment that was added later. Copyright basically couldn't exist as we know it without Fair Use. However, it is definitely subjective and always evaluated on a case-by-case basis, though there are often precedents set which help you understand the likely judgements of Fair Use before you're in court.

I was definitely being imprecise with my wording though, as I was thinking about a _transformative_ work when I read their attorney say "derivative" work, where a transformative work is a type of derivative work that is protected by [subjective] Fair Use.

I also agree with you though that I wouldn't even necessarily consider this a transformative work of the other map, as that assumes he started with the other map. But just starting with the same facts, which aren't copyrightable, and following standard cartographic conventions while exercising the author's own artistic expression into the final rendition would just be a separate work entirely.