First part would be creating a frame with scroll enabled, so you can move the area you're looking at and zoom only the "map" with ctrl+ or pinching, keeping the rest of the page normal. (edit: after checking, you still need JS to isolate the zoom level as far as i can tell)

Landmarks etc be added by creating fixed transparent elements with a group:hover popover, which displays more information.

You'll also be able to conditionally hide them with the pixel ratio pseudo-selector, depending on the zoom level. You'll also have to use transform to change their scale for readability.

Through anchor tags, you can even implement rudimentary jumping to positions in the map, as its nested in a frame.

I suppose you could use similar tricks as CSS-only chat to get the pan buttons to work without a page refresh