I think the same applies to the Mastodon or blog example (or Twitter, or Facebook) - to me, these are closer to broadcast publishing than to producing creative works, so copyright issues notwithstanding, I feel GP doesn't have full ownership on what they published on the public web either.

In this context, I feel it's OK for GP to remove their blog or Mastodon posts - not weird at all. However, I believe I am well within my rights to scrap GP's blog, screenshot all their Mastodon posts, and keep them for reference; I'd expect the blog to be indexed by the Internet Archive, and would find attempts at preventing the IA from making a copy as something between peculiar and antisocial.

That is, there's a distinction between owning the message itself, vs. owning its physical form or means of publishing. The former, I believe, is always shared between sender and recipient or recipients (up to and including everyone, if we're talking about a regular website without any access control). The latter is handled through normal property law mechanisms, but owning the medium only means you can decide who can access the message on/through that medium, not that you own the message itself.