The original author of busybox had a really good idea for this. I don't remember exactly but I think he partitioned the bootdisk in half amd had the bootloader fail a partition (and not boot it) if it didn't boot all the way. Each partition would apply OTA updates depending on which was active.