That would get you around 5% of the way in my opinion.

First part would be creating a frame with scroll enabled, so you can move the area you're looking at and zoom only the "map" with ctrl+ or pinching, keeping the rest of the page normal. (edit: after checking, you still need JS to isolate the zoom level as far as i can tell)

Landmarks etc be added by creating fixed transparent elements with a group:hover popover, which displays more information.

You'll also be able to conditionally hide them with the pixel ratio pseudo-selector, depending on the zoom level. You'll also have to use transform to change their scale for readability.

Through anchor tags, you can even implement rudimentary jumping to positions in the map, as its nested in a frame.