Also would make it easier to backup the program and its data or copy them to a new system.

But it seems because of the idea that some people may want to back up data and config separately, we end up with stuff scattered everywhere, with some even hidden away in the Windows Registry. Makes backups ridiculously complex.