Well, for me this is a question who owns the config file:

* If the package owns the file, and the user is not supposed to make changes to it, then package updates may do whatever to the file on update. It's good practice though, to leave a copy of the old file in /lost+found.

* If the user is allowed to make modification, then the user owns it. The configuration syntax becomes and API and should be treated as such. Breaking changes should be rare, preceded by a deprecation period and announced as major versions. As you say, it's questionable if such updates can/should be automated.

What about binary formats like gpg keyrings?

Easy, implement the config upgrade in the software (launcher) itself. 1. check config version on launch. 2. if old version upgrade.

Yes, ideally, every version of software that changes any external data format (e.g. of config files) should always accept both new and old versions.

But, unfortunately, some software does not, and it still needs to be packaged.

True. And the only effective solutions I can think of are extremely ressource intensive (man hours) and therefore unrealistic. (Lile wrapping non-compliant apps and using mandatory access control for enforcement)

And then you have moved stuff around and solved absolutely nothing, but you have created a new, permanent surface for security issues.