I have this conversation with artists sometimes. If I buy a print, yes, you retain copyright, but I expect an exemption when I post a picture of my print on Facebook, hanging in my living room. On the other hand, many artists want to retain the copyright of an original I purchase - which I usually disagree with. I can offer an exemption to my copyright which I purchase with the work that allows you to continue to sell prints. But if I'm going to pay a hefty price for an original work of art (or architecture) I expect to own it, copyright and all.
How about the 2nd amendment? Did they founders intent for folks to own machine guns?
My point was that calling anything an "absolute right" is problematic. Absolute allows no opportunity to assess intent, and the intent of a right of panorama is obviously not to allow rebroadcasting of soccer matches.
But the public owns it, so yes, it should be public domain.