I could also see them running a read-only system partition with snapshots for updates. And with btrfs send/receive you could do 'system updates' with just incremental snapshots, and the ability to rollback.

Or just use packages and make regular snapshots.