If a read replica fails, I'd expect no downtime (possibly a few errors as connections get cut off abruptly). Although there's always the risk that the remaining instances aren't able to handle the additional load.

If the master fails, you'll get a ~2min downtime